ON THE CLASSIFICATION OF HYPERBOLIC 
ROOT SYSTEMS OF THE RANK THREE. PART II 
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jrt . Abstract. Here we prove classification results announced in Part I (alg-geom/ 



9711032). We classify maximal hyperbolic root systems of the rank 3 having re- 
stricted arithmetic type and a generalized lattice Weyl vector p with p^ > (i.e. 
CO ' of elliptic or parabolic type). We give classification of all reflective of elliptic or 

^^ I parabolic type elementary hyperbolic lattices of the rank three. 



We apply the same method (narrow places of polyhedra) which was developed to 
^ \ prove finiteness results on reflective hyperbolic lattices. We also use some additional 

^f^ ' arithmetic arguments: studying of class numbers of central symmetries. 

C^ , The same methods permit to get similar results for hyperbolic type. We will 

consider hyperbolic type in Part III. 

These results are important for Theory of Lorentzian Kac-Moody algebras and 
I — ' some aspects of Mirror Symmetry. 



0. Introduction 

Here we prove results announced in Part I [N14]. We continue numeration of 
Sections started in Part I. We also keep notations of Part I. 

We consider main hyperbolic (i. e. of signature (1,/c)) lattices (i. e. non- 
degenerate integral symmetric bilinear forms) S of the rank three and with square- 
/\ ' free determinant d. Here "main" means that S should be even for even d. The 

(73 ■ lattice S is defined uniquely by its determinant d = det(5') and some additional 

invariant r] (see Sect. 2.2 of Part I or Sect. 3.2 here) where < ry < 2*, here t is 
the number of odd prime divisors of d. 

Let W{S) be the reflection group of S (generated by reflections in roots a E S 
with a^ < 0) and M. its fundamental polyhedron in hyperbolic space C{S) defined 
by S. We denote by A{M) = {</>€ 0+(5') | (j){M) = M} the group of symmetries 
of Ai. An involution u G A{A4) is called a central symmetry if u acts as a central 
symmetry in C{S). Two central symmetries from A{Ai) are called equivalent if they 
are conjugate in A{Ai). In Sect. 3, Theorem 3.2.1 we give a formula for the number 
h — hnr{S) of classes of central symmetries of the lattice S with invariants [d, rj). 
This formula uses the Legendre symbol and class numbers of imaginary quadratic 
fields. We use this formula to give the list of all the lattices 5" with h = hnr{S) < 1 
and d < 100000. This list was announced in Table 3 of Part I and contains 206 
invariants {d, ry) . This list is important for classification of the reflective lattices S. 

A hyperbolic lattice S is called reflective if there exist a non-zero p E S and 
a subgroup A C A{A4) of finite index such that A{p) = p. Here p is called a 
generalized lattice Weyl vector. If S has a generalized lattice Weyl vector p with 
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2 V.V. NIKULIN 

p^ > 0, then 5" is called reflective of elliptic type (or elliptically reflective). If S has 
a generalized lattice Weyl vector p with p^ = and does not have a generalized 
lattice Weyl vector p with p^ > 0, then S is called reflective of parabolic type. If 5" 
has a generalized lattice Weyl vector p with p^ < and does not have a generalized 
lattice Weyl vector p with p^ > 0, then S is called reflective of hyperbolic type. It is 
easy to prove that h < 1 ii S is reflective of elliptic type, h = if S is reflective of 
parabolic type and h = 0,2 if S is reflective of hyperbolic type and rk S" = 3. Thus, 
Table 3 contains all elliptically or parabolically reflective main hyperbolic lattices 
5" of the rank 3 and with square-free determinant d < 100000. 

In [N4], [N5], [Nil] and [N13] we proved flniteness results about reflective hy- 
perbolic lattices using some geometrical arguments: studying of narrow places of 
fundamental polyhedra. In Sect. 4 we improve and optimize this method for 2- 
dimensional case to apply it for classiflcation of reflective hyperbolic lattices of rank 
3. In Sect. 4.3 we apply these results to fundamental polygons of the reflection 
groups of reflective hyperbolic lattices of elliptic or parabolic type and of rank 3. 
As a result, we get estimates of the determinant and some other invariants of these 
lattices. 

In Sect. 5 we apply results of Sects. 4 and 5 to classify all elliptically or parabol- 
ically reflective hyperbolic lattices of the rank 3 and with square- free determinant. 
The list of these lattices was announced in Tables 1 and 2 of Part I (see [N14]). In 
particular, we prove that all elliptically or parabolically reflective main hyperbolic 
lattices of the rank 3 and with square-free determinant are contained in Table 3. 
To study reflective type of lattices, we use Vinberg's algorithm [V2] . 

The same methods as developed here permit to classify reflective hyperbolic 
lattices of the rank 3 of hyperbolic type. We hope to do this in Part III. 

3. The number of classes of central symmetries of main hyperbolic 
lattices with square-free determinant and of the rank 3. 

3.1. Reminding of some classical results about binary positive lattices. 

Here we remind some classical results about binary positive quadratic forms (e. 
g. see [B-Sh] and [C]). 

We consider binary (i.e. of the rank two) positive deflnite lattices K with square- 
free determinant d = det K. They will be called fundamental. If d is odd, the 
fundamental lattice K is unimodular over Z2. Considering K over Z2, it is easy to 
see that K is odd if (i = 1, 2 mod 4. If d = —1 mod 4, the lattice K may be odd 
or even. The number 

( — det K, if K is even, 
D=\ 3.1.1 

\ -4 det K, if K is odd ^ ^ 

is called discriminant of a binary lattice K. Discriminants of fundamental binary 
lattices are called fundamental discriminants. Thus, for a fundamental discriminant 
D either D = 1 mod 4 or D = ±4, 8 mod 16. A binary positive lattice K is called 
classical fundamental ifd = det K is square- free and additionally K is even ifd= —1 
mod 4. The discriminant D of a classical fundamental binary lattice K is called the 
classical fundamental discriminant. Thus, for a classical fundamental discriminant 
D either D = 1 mod 4 (when K is even) or D = —4, 8 mod 16. 
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Theorem 3.1.1. The number h{D) of proper classes (i. e. classes of preserving 
orientation isomorphisms of oriented lattices) of classical fundamental binary lat- 
tices K of the discriminant D < is equal to the class number of the imaginary 
quadratic field Q(V^) of the discriminant D and is given by the Dirichlet's formula 



MI^) = 7^ >. - r (3.1.2) 



w 
2D\ 
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0<r<|D| ^ 
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ifD = -?,, 


4, 


ifD = -A, 


. 2, 


otherwise, 




where 

w= I A, ifD = -A, (3.1.3) 

y 2, otherwise, 

and (-^) is the Kronecker (i.e. generalized Leg endre) symbol. 

Let K be an odd binary lattice of an odd square- free determinant d where d = ~1 
mod 4 (i. e. it is fundamental, but it is not classical fundamental). The maximal 
even sublattice of K is the lattice T(2) where T is the fundamental (even) lattice 
of the determinant d. Thus, T(2) C -ftT is an odd overlattice of index 2 of the 
lattice T(2). li d = —1 mod 8, then the overlattice K is unique. If d = —5 
mod 8, then there are 3 overlattices T(2) C K (which we should consider up to 
proper automorphisms of T to get the number of proper classes of K) . We have 
discr K = —Ad and denote h{—Ad) the number of proper classes of lattices K of 
that discriminant. By considerations above, we get 

if D = 1 mod 8, 
h{AD) = { 3h{D), ifD = 5 mod 8 and D < -3, (3.1.4) 

if D = -3. 

Let K he a, fundamental positive binary lattice of the discriminant D. Here D = 
0, 1 mod 4. li D = mod 4, then D = ±4, 8 mod 16. We use the discriminant 
form technique [Nl]. The genus of K is defined by the discriminant form of K (it 
is quadratic if K is even, and is bilinear if K is odd) which is defined by the map 
/U : p H^ /Up G {0, 1}, p runs through all odd prime p\D. For odd (prime) p\D, the 

discriminant form b^p = bg{p) where i^-) = (—1)'^''. The form bK2 is trivial if 

either D = l mod 4 or D = 4 mod 8. If D = 8 mod 16, then 6^2 = ^f ^(2). 

We denote by t the number of prime divisors of D. If D = 1 mod 4 (equivalently, 
K is even), existence of K with the invariants {D, n) is equivalent to 

^[(l-p)+4//p] = 2 mod 8, (3.1.5) 

p\d 

and there are 2*~^ genuses. 

If D = —A mod 16, (equivalently, d = 1 mod 4) existence of K with the invari- 
ants (D, n) is equivalent to d = 1 mod 4 which is given. Thus, number of genuses 
is equal to 2*~^. 

If D = 4 mod 16 (equivalently, d = —1 mod 4 and the lattice K is odd). 
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discriminant D — —d (by the construction above), and number of genuses is equal 
to 2*-2. 

If D = 8 mod 16, there are no conditions on [D, ^) for existence of K, and 
number of genuses is equal to 2*~^. 

Thus, the number of different genuses of the discriminant D is equal to 2'^^^^ 
where 

' t-1, if D = 1 mod 4, 

r(D) = <(t-l, if D = -4, 8 mod 16, (3.1.6) 

t-2, if D = 4 mod 16, 

and t is the number of all different prime divisors of D. It is known that each genus 
contains the same number of classes. Thus, number of classes h{D)g in a genus of 
a fundamental lattice K is equal to 

KD). = ^. (3.1.7) 

A fundamental binary lattice K is called ambiguous iiO{K) contains a reflection 
(equivalently, the oriented lattice K is proper equivalent to the lattice K with the 
opposite orientation). If 0{K) contains a reflection in 5i G -ftT where 5i is primitive, 
then it contains reflection in a primitive 82 & K which is orthogonal to ^i. We can 
suppose that ni = 5f < n2 = 5|. It follows that a reflective lattice K is generated 
by either {5i, ^2} or {5i,52, {81 + 62) /'2.}. We say that the type is I if we have 
the flrst possibility, and the type is II if we have the second one. It is easy to 
see (using classiflcation of 2-dimensional reflection groups) that ni < 77-2 and type 
are invariants of the ambiguous class K except K = (1,1;/) — (2,2;//) of the 
discriminant —4. Except this case, two ambiguous binary lattices are isomorphic 
iff they have the same invariants (ni, n2; type). We denote by /ir(ni, n2; type) the 
ambiguous positive lattice with invariants (ni, 77-2; type). A lattice K{ni,n2; I) 
exists for any natural ni, 712 G N. A lattice K{ni,n2]II) exists iff ni,n2 G N, 
m = n2 = mod 2 and ni + ?i2 = mod 4. 

We have det K{ni, 712] I) = nin2, and the ambiguous binary lattice K{nij 712] I) 
is fundamental if and only if both rii and 712 are square-free and (ni, 712) = 1. This 
lattice is odd of the discriminant D = —47ii7i2- 

We have det K {711,712; II) = 711712/4, and K {711,712; 1 1) is fundamental iff both 
ni, n2 are square-free and (ni, 712) — 2. The lattice K{7ii,7i2; II) is even iff ni-|-n2 = 
mod 8. Thus, we have 

\ -711712/4, if m-t- 772 = mod 8 
D{K{7ii,7i2;II)) = < 

y —ni7i2, otherwise. 

It follows that the number hr{D) of ambiguous classes (proper or improper, it 
does not matter because they are improper equivalent to itself) of the discriminant 
D is equal to 

r 2^-\if D = 1 mod 4, 

2^-\if D = -4 mod 16, 

I 2*-2,if L*- ^ ^ ^^ ^- ^ 



hr{D) = 2"(^) 



4 mod 16, 
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This number is equal to the number 2'^^^^ of genuses because ambiguous lattices 
correspond to elements of order 2 in the group A of classes of discriminant D. 
We denote this group as A2. For a class a E A, the class —a G A is the class 
which is improper equivalent to a. The group of genuses is A/2A, and natural 
homomorphism A -^ A/2A is the genus map. 

Further we will be especially interesting in genuses which do not contain more 
than one class of non-ambiguous lattices with respect to the general (i.e. proper 
or improper) equivalence. A genus is called ambiguous if it contains an ambiguous 
lattice. Otherwise it is called non- ambiguous. 

Using the genus homomorphism, it is easy to see that a discriminant D contains 
a non-ambiguous genus iff 2'^^^'^^\h{D). If 2'^^^'^^\h{D), the number of classes of 
general equivalence (i.e. proper or improper) in a non-ambiguous genus is equal to 



HP) 



Thus, we have 



2t(C)+i • 

Lemma 3.1.2. There exists a non-ambiguous genus of the discriminant D iff 
2'^(P')+'^\h{D). The number of classes of the general equivalence (i.e. proper or 
improper) in a non-ambiguous genus is equal to h{D)/2'^^^'^^ . In particular, a 

non-ambiguous genus contains exactly one class of general equivalence iff h{D) = 

2t{d)+i_ 

Number of ambiguous classes in an ambiguous genus is equal to ^2 A n A2. By 
elementary considerations with finite Abelian groups, we get 

Lemma 3.1.3. An ambiguous genus contains only ambiguous classes iff h{D) = 
2"" where m < 2t{D), and the class group A ^ (Z/4)"^-^(^) © (Z/2)2^(^)-"^. 
Equivalently, h{D) = 2"^ where m < 2t{D), and the number of ambiguous classes 
in the principal genus 2A is equal to 2"^~'^(^) . 

Lemma 3.1.4. An ambiguous genus contains exactly one non-ambiguous class iff 
either h{D) = 2^^^^ ■ 3 or h{D) = 2^^^)+^ and A = (Z/8) © (Z/2)^(^)-^ We 
remark that if h{D) = 2'^(^)+^, then either D is of the type of Lemma 3.1.3 (when 
an ambiguous genus contains ambiguous classes only) or A = (Z/8) © (Z/2)'^*^'^)~-'^ 
and an ambiguous genus contains only one class of general equivalence of non- 
ambiguous lattices. The last case is characterized by the property that the principal 
genus 2A contains exactly two ambiguous classes. 

Below we calculate numbers hnr{D,n), hrj{D,n), hrjj{D,n) and hr{D,fx) of 
non-ambiguous classes of general equivalence, ambiguous classes of type /, ambigu- 
ous classes of type // and ambiguous classes respectively of the genus (D, /S). We 
have 

/ir(-4, ^) = hri{-A, fx) = hrii{-A, fx) = 1 and /inr(-4, //) = 0. (3.1.9) 

If D ^ —4, we have 

hr{D, fi) = hri{D, ^) + hrn{D, ^) (3.1.10) 

and 

hnr{D, /j) = (/i(D)/2^(^) - hri{D, /j) - hrii{D, n))/2. (3.1.11) 

Below we calculate hri{D, y) and hrii{D, fx). Since the principal genus pr is am- 
biguous, for the number of non-ambiguous classes in an ambiguous genus ambig we 
get 

1 .('7-1 „,™,I>„■„^ /u/-nt\inT{D) u^ I T^ „„^ j,„ /' 7-1 „„^^ /o lo -\ 1o^ 
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We remind (Lemma 3.1.2) that for a non-ambiguous genus nambig we have 

[0, if2"(^)+i //i(L>), 

hnriD,nambig) = i ^^ if 2-(^)+i|/i(I)). ^^'^'^^^ 

Case D = 1 mod 4. Then the determinant d = —D = —1 mod 4. All ambigu- 
ous classes of the genus (D, fx) are given by K{2di, 2d2] II) such that did2 = d and 
/2c|/p\ ^ ^_^^^^ .f ^1^^ ^^^ (^^) = (-1)''^ if p|rf2. Thus, 

hrii{D,^) = ^{di\d I rfi < d/di L l'^Ml\ = {-lyp Vp|rfi 



P ^ (3.1.14) 

and 

hri{D,n) = 0. (3.1.15) 

The principal genus pr is given by the ambiguous lattice K{2, 2d; II). All ambigu- 
ous classes of that genus are K{2di,2d2] II) such that did2 — d and i — ^ j = 
j^2di^j if p|rfi and (^) = (^Hf^i^) if p|rf2. Thus, 

hrn{D,pr) = ^{di\d \ di < d/di & f^M^j = i Vp|di & ( ^ j = 1 yp\{d/di)} 

(3.1.16) 
and 

hri{D,pr)=0. (3.1.17) 

Case D = 4 mod 16. Then d = —D/4 = — 1 mod 4. All ambiguous classes of 
the genus {D, n) are given by K{di, d2] I) such that did2 = d and f -^-^ j = (— l)'^p 

if p|rfi and (^j = (-l)'^p if p|rf2- Thus, 

/ir7(L',;u) = S{rfi|rf I di<d/diL ( ^^ J = (-1)^'' Vp|rfi 

&f^^^^^^l=(-l)-Vp|(c/M)}, 



^ ^ (3.1.18) 

and 

hrii{D,fi)=0. (3.1.19) 

The principal genus is given by the ambiguous lattice K{l,d;I). All ambiguous 
classes of that genus are K{di, d2] I) such that did2 = d and i ^-p j = i ^ ^'^ j if 

p\di and (^) = (^) if pM2. Thus, 
hri{D,pr) = ^{di\d I rfi < d/di & /^^^ J = 1 Vp|rfi & ( — j = 1 yp\{d/di)}, 

(o 1 o^^ 
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and 

hrn{D,pr) = 0. (3.1.21) 

Case D = —4 mod 16. Then d = —D/4 = 1 mod 4. All ambiguous classes 
of type I of the genus (-D,/u) are given by K{di,d2]I) such that did2 = d and 
(^\ = (-l)Mp iip\d^ and (^\ = {-lY^ \ip\d2. Thus, 

hri{D,^) = ^{di\d I di < d/di & (^hlP\ = (_i)Mp vp|rfi 

\ P J (3.1.22) 

All ambiguous classes of the type II of {D, y) are given by K{2di, 2d2; II) such that 
did2 = d and (^^) = (-1)^^ if p\di and (^^) = (-1)^^ if p\d2. Thus, 



P ^ (3.1.23) 

The principal genus is given by the ambiguous lattice K{l,d]I). All ambiguous 
classes of the type I of that genus are K{di,d2;l) such that did2 = d and ( -^-^ ) = 

(^) if pirfi and (^) = (^) ifp\d2. Thus 

hri{D,pr) = ^{di\d I rfi < rf/rfi & /"^ J = 1 Vp|di& ( — J = 1 \/p\{d/di)}. 

(3.1.24) 
All ambiguous classes of the type II of the principal genus are K{2di, 2d2; II) such 

that did2 = d and (^) = (^^) if p\di and (^) = (^^if^) if p\d2. 

Thus 

hrii{Djpr) = 

m\d I * < <*/* & (^) ^ (^) V.I* & (^) ^ (i) v.i(d/*)}.^^^ 

Case D = 8 mod 16. Then (i = —D/4 = 2 mod 4. All ambiguous classes of the 
genus (D, //) are given by K{di, 2d2] I) such that did2 = d/2 and ( -^-^ ) = (— 1)'^^ 

i{p\di and (^^) = (-1)^^ if p\d2. Thus, 

/ir,(A^) = ^{di\d/2 I & r^") = (-1)^- yp\di 

^fmyi) = ^_rr.y,\^d/2d,)i 
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and 

hrniD,n)=0. (3.1.27) 

The principal genus is given by the ambiguous lattice K{1, d; 1). All ambiguous 
classes of that genus are K{di, 2d2] I) such that did2 = d/2 and ( -^-^ j = ( — L^/£ 



iip\di and (^) = (^Hf^i^j if p|rf2. Thus, 



hri{D,pr) = t{d^\{d/2) \ ( ^"j = 1 Vp|rfi & ("^ ) = 1 Vp|(rf/2rfi)} (3.1.28) 



and 



hrniD,pr) = 0. (3.1.29) 



Using considerations above, In Appendix I, we give Program h2 for "GP/PARI" 
calculator which for a fundamental discriminant D < and the genus {D, ^) cal- 
culates the vector 

ihri{D, ij), hrii{D, ^u), hnr{D, n)). (3.1.30) 

The "GP/PARI" calculator uses the Shank's method [Sh] to calculate the class 
numbers h{D) of the discriminant D. It is very fast: 0{\D\^/^) operations. We 
code the invariant fj. by the non-negative integer // having the binary decomposition 

/^ = ^Pfc^Pfc_i •••^Pi (3.1.31) 

where pi, . . . , pk are all odd prime divisors of D in increasing order. 

3.2. The number h of non-reflective classes of central symmetries of main 
hyperbolic lattices of the rank 3. 

We denote by 5" a main hyperbolic (i.e. of the signature (1,/c)) lattice with 
square-free determinant and of the rank 3. Remind that main means that the 
lattice S is even if the determinant d = det(5') is even. If d is odd, then S is 
necessarily odd. 

By Proposition 2.2.4, the lattice 5" is defined by its invariants {d, r]) where d = 
det(5') and the invariant ry is the map ?7 : p i-^ ?7p G {0, 1} of the set of all odd prime 
divisors of d. The ry is defined by the condition 

^5,=6?;(P), (7)=(-ir^- (3-2.1) 

Here qs and hs denote discriminant forms of the lattice S. Here and in what follows 
we use discriminant forms technique and notations in [Nl] . We will especially often 
using theorems of existence of a lattice with a given discriminant form (Theorems 
1.10.1 and 1.16.5 in [Nl]). 

For even d the lattice 5" is even and the discriminant quadratic form qs^ = qg (2) 
where 6*2 = ±1 mod 4. We denote 

62 mod 8 = ±1 mod 8 if ^2 = ±1 mod 4. (3.2.2) 

We have 

^ [{I - p) + Ar]p] + 62 mod8 = -l mod 8. 
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Here sign qj' (p) = {1 — p) + Arjp mod 8 and sign q^ '{2) = 62 mod 8. Since 62 
mod 8 = ±1 mod 8, we get 

Y^ [{1 - p) + 4r]p] = 0, -2 mod 8 (3.2.3) 

odd p\d 

and the invariant $2 is defined by 

02 mod 8 = - Y^ [{l-p)+ 4r]p] - 1 mod 8. (3.2.4) 

odd p\d 

Here (3.2.3) is the condition of existence of a main hyperbohc lattice S with the 
invariants ((i, rj) if d is even. If d is odd, the lattice 5" is odd, and there are no 
condition of existence of S. It always does exist. Thus, for any square-free natural 
number d and any map rj of the set of all odd prime divisors of d into {0, 1}, there 
exists a main hyperbolic lattice S of the rank 3 with invariants {d, rf) if and only if 

Y [(l-p)+4?7p] = 0, -2 mod 8 (3.2.5) 

odd p\d 

if d is even. 

We consider primitive elements f E S such that n = /^ > 0. We consider them 
up to ±/, thus we can suppose that / G V~^{S) where V~^{S) is the light-cone of 5" 
(see Sect. 1.1). We consider the elements / such that there exists an automorphism 
Uf of S which is identical on / and is —1 on the orthogonal negative definite lattice 
K = f^. Then 

either S = Zf ® K or S = [Zf ® K, (/ © k)/2], keK. (3.2.6) 

This automorphism is called a central symmetry of S. Geometrically, it is the 
central symmetry at the point ffi++/ of the hyperbolic space C{S) = V^{S)/M.++ 
defined by S. Vice versa, any (j) G 0~^{S) acting as a central symmetry in C{S), has 
the form (p = Uf. By considerations over Z2, one can see (using (3.2.6)) that either 
n is odd or n = 2 mod 4. (One can also use that Uf : x \-^ —x + (2(x, f)/ f'^)f, 
xe S, and f'^\2{S, /).) Let n = 2^ni where (ni, 2) = 1, k = 0, 1. Then by (3.2.6), 

ni|rf, and /c = 1 if rfiseven, and [ ^ J = (-1)''^ V odd p|n. (3.2.7) 

The central symmetry Uf is called reflective if there exists a reflection S5 of 5" in 5 _L 
/ (i.e., S G K). Otherwise, Uf is called non-refiective. Geometrically, the central 
symmetry Uf is reflective if and only if the center M_|__|_/ belongs to a mirror of the 
reflection group W{S). We want to calculate the number hnr{S) of non-reflective 
central symmetries Uf of S up to conjugation in 0(5") (i.e., the number of non- 
reflective classes of the central symmetries). Since ±1 belongs to the center oiO{S), 
it is sufficient to consider Uf up to 0+{S) = {(j) E 0{S) \ (j){V+{S)) = V+{S)}. 
Acting by the reflection group W{S), we can always suppose that the center IR++/ G 
A4 where A^ is a fundamental polyhedron of W{S) (see Sect. 1.1). If Uf is non- 
reflective, it is equivalent that Uf G A{P{M)pr) = {</> G 0+{S) \ (p{M) = M}. 
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if they are conjugate by A(P{A4)pj.). Thus, the number h = hnr{S) of classes of 
non-reflective central symmetries of 5" is the same invariant h = h{S) of S which we 
have introduced in Sect. 2.3. The central symmetry Uf is defined by the element 
/ (it is defined by Uf up to ±/). Thus, it is sufficient to study all these / up to 
action of 0(5"). Obviously, (n, K) are invariants of /. Here K = f^ is a, negative 
binary lattice which we consider up to isomorphism. 

First, we calculate invariants of genus of the central symmetries Uf,fES above. 
Here two elements fi E S and f2 ^ S have the same genus if they are conjugate 
over Zp for all prime p. Then we calculate the number of non-refiective classes Uf 
using class numbers of binary positive lattices. We had considered them in Sect. 
3.1. 

Case I: d = 1 mod 2, equivalently 5" is odd. Then there are two cases. 

Case (I, n): K = f^ is even. Then n is odd because S is unimodular odd over 
Z2, and then / is a characteristic element of 5" (i.e. (/, x) = (x,x) mod 2 for any 
X G S). By (3.2.6), we then have S = Z/ © K. It follows that n\d, det K = d/n, 
and the discriminant quadratic form qk , p\{d/n), is equal to 

QK, = qs, = qi'Jip), (^) = (-l)''^ P\{d/n). 
By discriminant form technique [Nl] , existence of K is equivalent to 

Y^ [(1 -p)+ 4?7p] = -2 mod 8. (3.2.8) 

p\{d/n) 

Conditions (3.2.7) and (3.2.8) are equivalent to existence of Uf with /^ = n. The 
lattice K{—1) is an even fundamental positive binary lattice of the discriminant 
discr K = ~ det K = —d/n. Thus, —d/n = 1 mod 4. The genus of K{—1) is equal 

to {—d/n, e{p) + rip). Here ( ^^ j = (— 1)*^*^^^ and it is known that e{p) = {p— l)/2 

mod 2. To be shorter, here we denote by e{p) + r]p the map p \-^ e{p) + rjp where 
p runs through all odd prime p\{d/n). It follows that the number of non-refiective 
classes of central symmetries f E S of the type (I, II) is equal to 

hnrii{S) = y^ hnr{—d/n,e{p)+rip), (3.2.9) 

rL\d & (3.2.7) 
& (3.2.8) 

where we consider the sum by all n such that n\d and (3.2.7), (3.2.8) are valid. 

Case (I, l):K = f^is odd. By (3.2.6), (3.2.7), then n\2d and det K = (n, 2fd/n, 
discr K = —4{n,2)'^d/n. It follows that K{—1) is a fundamental positive binary 
lattice. 

For odd p\ det K, we have 



and 



qK,=qs,=q'fJiPl (7) = ("1) 



r 0, if n is odd. 
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If n is odd, existence of K is equivalent to 

-2 modSG Y^ [{l-p)+4r]p]+4uj{d/n) + {0, ±2} mod 8. 

p\{d/n) 

It is equivalent to 

Y^ [{l-p)+4r]p]+4u{d/n)^2 mod 8. (3.2.10) 

p\(d/n) 

Here uj{k) = (/c^ — l)/8 mod 2, it is known that ( — ) = (— l)'^*-^-'. For even n there 

are no conditions of existence of K. The lattice K does always exist. The genus of 
K{—1) is equal to (— 4(n, 2)'^d/n, e{p) + r]p). 

Thus, this case is characterized by the condition: d is odd, n\2d, we have 
Z/ ® K d S where K is an odd fundamental binary lattice of the discriminant 
— 4(?i, 2)^(i/n. If n is odd, Z/ ® K = S. If n is even, the lattice S is generated by 
Z/ © K and u = {f + k)/2 where k & K is a primitive element with the property: 
[k, K) = mod 2. The element k mod 2K, and the overlattice 5" are defined 
uniquely. For both these cases, any automorphism of K can be extended to the 
automorphism of 5" identical on /. It follows that the number of no n- reflective 
classes of / of the type (I, I) is equal to 

hnri{S) = y^ hnr{—4d/n,e{p) + rip) + y^ hnr{—16d/n,e{p)+rip). 

n\d & (3.2.7) n = 2ni\2d 

& (3.2.10) &(3.2.7) 

(3.2.11) 
As a result, we get that for the case I (equivalently, when d is odd) the full 
number of non-reflective classes of central symmetries / G 5" is equal to 

hnr{S) = y^ hnr{—d/n,e{p)+rip) 

n\d & (3.2.7) & 
(3.2.8) 

+ y^ hnr{—4d/n,e{p) + rip) + >^ hnr{—16d/n,e{p) + rip). 

rL\d & (3.2.7) n = 2ni|2d (3.2.12) 

& (3.2.10) & (3.2.7) 

Case II: d is even, equivalently, the lattice S is even. Then n = 2ni where 
ni\{d/2). This case is also divided in two cases: 

Case (II, II): the lattice K = f^ has an odd determinant. By (3.2.6), we have 
S = Zf®K. Then qs^ = 9^/2(2) and m = n/2 = 62 mod 4. By (3.2.4), 

n/2 = - Y [(1 -P)+ "^Vpl - 1 mod 4. (3.2.13) 

odd p\d 

We have det-ftT = d/n and, for p\{d/n), 

Existence of K is equivalent to X]p|(d/n) [(-'- ~ P) + ^Vp] = ^2 mod 8. This follows 
from the condition (3.2.2) (or (3.2.3)) of existence of S" and (3.2.7), (3.2.13). Thus, 
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Thus, this case is characterized by the condition: d is even, n\d is even, S = Z/© 
K, where K{—1) is an even fundamental binary lattice of the genus {—d/n, e{j>) + 
r]p). Thus, the number of non-reflective classes / of the type (11,11) is equal to 

hnrii{S) = y^ hnr{—d/n,e{p)+rip). (3.2.14) 

n = 2nj^\d & (3.2.7) & 
(3.2.13) 



Case (11,1): the lattice K = f^ has even determinant. The discriminant form 



^Ifh = ^SU^)- By (3.2.6), we have 



QK2 = q: 



i^) (2)©,g)(2). 



It follows that the lattice K{l/2) is odd fundamental, K has determinant 4d/n, 
and for p\{d/n) one has 



?i., = ?5, = <^(p), (y) = (-ir^- 



Existence of the lattice K is equivalent to existence of the lattice S. The odd 
fundamental binary lattice K{—l/2) has the determinant d/n, and the discriminant 
— 4(i/n, and for p\{d/n) one has 

Thus, for this case, d is even, /^ — n\d is even, K = T(— 2) where T is an odd 
fundamental positive binary lattice of the discriminant —4d/n and of the genus 
(— 4(i/n, e(p) +uj{p) + rjp). The lattice 5" is an overlattice Z/ (B K C S of the index 
2 generated by (/ + k)/2 where k E K satisfles (/c, k) = —n mod 8. If d/n = —1 
mod 4, then k mod 2K and 5" are unique. If d/n = 1 mod 4, there are exactly 
two different elements k which give the same lattice S. If discr T = —4, elements 
k mod 2K are conjugate by 0{K). If discr T < —4, elements k mod 2iir are not 
conjugate and give different classes of f E S. Further we consider two cases: 

Case d/n = —1 mod 4. The number of such non-reflective classes f E S of the 
type (II, I) is equal to 

hnri^-i{S) = y^ hnr{—Ad/n,e{p)+uj{p)+rip). (3.2.15) 



n = 2ni \d & (3.2.7)« 
ni= — d/2 mod 4 



Case d/n = 1 mod 4. Then the class f E S is non-reflective if and only if the 
lattice K does not have a reflection which can be extended identically on Z/ to 
give an automorphism of 5". If K is non-ambiguous, then 0{K) = {±1}, and we 
then get two classes of / corresponding to two different choices of k mod 2K. If 
d/n 7^ 1 and T is ambiguous of the type II, then 0{K) has order 4, only ±1 G 0{K) 
preserve the element k mod 2K, and reflections of K change places two possible 
different elements k mod 2K. Thus we get exactly one non-reflective class f E S. 
If d/n 7^ 1 and T is ambiguous of type I, then reflections of T are identical on 

T^/nr^ . J „. — 1 i- 1 — ] — o ;j — j-;„„iu. — J^ rpu,,„ j-U„,. „;, a — .j-;, .!„„„„„ 
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f E S. If d/n = 1, the lattice T = K{lj 1; /) = K{2, 2; //), and it has reflections of 
both types I and II. This case does not give non-reflective classes f E S. Thus, for 
this case, the number of non-reflective classes f E S of type (II, I) is equal to 

hnri^i{S) = 

22 [2hnr{-4d/n, e{p) + uj{p) + rjp) + hrii{-Ad/n, e{p) + u{p) + rjp)]. 

n = 2ni\d & (3.2.7) & (3.2.16) 

n-^'=d/'2 mod 4 & n<_d 

As a result, we get for the case II (equivalently, when d is even) that the full 
number of non-reflective classes of central symmetries f E S is equal to 



hnr{S) = y^ hnr{—d/n,e{p)+rip) 

n = 2n-^\d & (3.2.7) & 
(3.2.13) 

-|- 2. hnr{—4:d/n,e{p) + uj{p) + rip) 



n. = 2ni \d & (3.2.7)& 
n\^ — d/2 mod 4 

+ ^ [2hnr{~4d/n, e{p) + uj{p) + rjp) + hrii{-Ad/n, e{p) + uj{p) + rjp)]. 

n = 2ni\d & (3.2.7) & (3.2.17) 

n-^^d/2 mod 4 & n<d 

As a flnal result, we get 

Theorem 3.2.1. Let d be a square-free natural number and rj : p \-^ {0, 1} a 

map of all odd prime divisors p\d into {0, 1}. Then there exists a main hyperbolic 
lattice S of the rank 3 with the square-free determinant d and the invariant rj (see 
(3.2.1)j if and only if for the even d the congruence (3.2.5) is valid. The number 
h = hnr{S) = hnr{d, rj) of classes of non-reflective central symmetries of the lattice 
S with the invariants {d,rj) is given by (3.2.12) for the odd d and by (3.2.17) for 
the even d. 

Below we will code the invariant rj by the non-negative integer rj having the 
binary decomposition 

V = Vpf-Vpi (3.2.18) 

where pi, . . . ,pt are all odd prime divisors of d in increasing order. 

In Appendix: Programs, we give the Program 2: h3 for "GP/PARI" calculator 
which using Theorem 3.2.1 and Program 1: h2 (see Sect. 3.1) calculates the in- 
variant h = h{S) = hnr{d,rj) if a main hyperbolic lattice S of the rank 3 with the 
invariants {d, rj) does exist (otherwise, the result will be unreasonable) . Using the 
flrst statement of the Theorem 3.2.1 and the Program 2, we give Program 3: refh3 
which gives all pairs of invariants {d, rj) such that d < N, there exists a main hyper- 
bolic lattice of the rank 3 with the invariants {d, rj) and the invariant hnr{d, rj) < 1. 
Using Program 3, we found all these pairs (d, rj) such that d < 100000. The result 
is given in Table 3 (Part I) and contains 206 lattices. Thus, we get 

Theorem 3.2.2. Table 3 (Part I) gives the complete list (it has 206 lattices) of 
main hyperbolic lattices S with square-free determinant d < 100000 and of the 
rank 3 such that the invariant hnr{S) < 1. In Table 3 we give invariants d, rj, 
h = hnr{S), the matrix of the lattice S and the reflective type of the lattice S . 

The greatest d of lattices S of the Table 3 is equal to 4466 in spite we did 
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Conjecture 3.2.3. Table 3 (Part I) gives the complete list of main hyperbolic 
lattices S with square-free determinant and of the rank 3 such that the invariant 
hnr{S) < 1. 

In Sect. 5, we shall use Theorem 3.2.2 to find all {d,r]) corresponding to el- 
liptically or parabolically refiective hyperbolic lattices S since for elliptically or 
parabolically reflective hyperbolic lattices the invariant h < 1. 

Similarly we calculated all pairs (d^rj) such that hnr{d,rf) = 2 and d < 100000. 
(One should change in two places of Program 3 h < 1 by h = 2.) The list contains 
259 pairs {d.rj). The last 10 pairs having the largest d are: (4290,1), (4326,2), 
(4902,4), (4991,7), (5226,0), (5334,2), (6006,2), (7590,8), (10374,2), (29526,2). 
It is very likely that this list also contains all pairs [d, i]) with h = 2. We shall 
use the list of these 259 lattices and lattices of Table 3 to flnd in Part III all 
hyperbolically reflective main hyperbolic lattices of the rank 3. All of them must 
have the invariant h — 0, 2. 

4. Narrow places of elliptic and parabolic 

convex polygons on hyperbolic plane, types of 

POLYGONS. Application to reflective lattices 

4.1. Narrow places of elliptic convex polygons on the hyperbolic plane. 

We remind (see Sect. 1.1) that a convex polyhedron in a hyperbolic space is 
called elliptic if it is a convex envelope of a flnite set of points (some of them at 
inflnity) and it is non-degenerate. In this section we shall consider only elliptic (i.e. 
ordinary flnite) convex polyhedra and often shall omit the word "elliptic" . 

Here we follow the general method (of narrow places of polyhedra) suggested 
in [N4], [N5] for proving flniteness results about arithmetic reflection groups in 
hyperbolic spaces. On the other hand, we shall prove much more delicate and 
exact statements, which are important for exact classiflcation. Our estimates in 
[N4] , [N5] were universal, they did not depend on angles of fundamental polyhedra. 
Here we get estimates which depend on angles of polyhedra which makes the narrow 
places of polyhedra method much more efficient. Our estimates here are optimal, 
we belive that one cannot impove them. 

We restrict by 2-dimensional case of narrow places of elliptic polygons on the 
hyperbolic plane, but all results can be easily generalized (like in [N4], [N5]) on 
elliptic polyhedra of arbitrary dimension in hyperbolic spaces. 

We shall often use the following trivial but important for us statement (certainly, 
it is well-known): 

Lemma 4.1.1. ^^^^ < - if < x < n/2 and x < v. 

Proof. The function sinx/x is decreasing if < x < 7r/2. In particular, 1 > 
sinx/x > 2/7r if < x < 7r/2. It follows the lemma for < x < y < -k 12. If 
< X < 7r/2 < y, we get 

siny 1 1 7r/2 y 

sinx sinx x{2/ti) x x 

This proves the statement. 
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Lemma 4.1.2. Let {AB) and {CD) are two lines on a hyperbolic plane with ter- 
minals A, B, C , D at infinity, and O a point on the hyperbolic plane which does 
not belong to each line [AB] and [CD] and orientations of the triangles AOB and 
COD coincide. We consider angles 9i = AOB, 62 = COD and 612 = BOC . Let 61 
and 62 are orthogonal vectors with square —2 to lines (AB) and (CD) respectively 
such that O is contained in both half-planes H'^ and Til . 



Then 



sin ^i±^ sin ^2±h2 
sin -^ sin -^ 



^45 a corollary, we get : 

1) If lines [AB] and {CD) do not intersect each other, then 



sin ii±^ sin ^^Mi^ 
2coshp= (5i,52) =4- 2 2 



sin -J sin -^ 

where p is the distance between lines {AB) and {CD) (here and in what follows we 
normalize the curvature k = —1). 

2) If lines {AB) and {CD) define an angle a containing O, then 

CJTI ^1~^21 ^\y^ 62—621 



2 COS a = (5i,52) =4 /. , ' 2 



sin ^ sm ^ 



where ^21 = -O12 = COB. 

Proof. We can correspond to two lines {AB), {CD) and a connected component 
(containing O) of the complement to these two lines in the hyperbolic plane two 
invariants up to motions of the hyperbolic plane. First invariant is equal to (5i, ^2) 
and uses Klein model of the hyperbolic plane. Second invariant is equal to the 
cross ratio [A : D : C : B] where we suppose that orientations of the triangles ABO 
and CDO coincide. This invariant uses Poincare model of the hyperbolic plane. 
On the other hand, it is clear that any of these two invariants defines the triplet 
{{AB), {CD), the connected component to their complement (containing O)) up 
to motions of the hyperbolic plane. It follows that there exists a function f(x) such 
that {61, 62) = f{[A: D :C : B]). One can check that f{x) = 4x - 2. 

Below we shall consider an equation 

{u — x){v — x) = auv (4-1.1) 

where u,v > and < a = cos^ ^ = (1 + cosq;)/2 < 1. Equivalently we have the 
equation 

x"^ — {u + v)x + uv{l — a) = 0. 

Its smallest root x = g{a,u,v) equals 



u + v- y/a{u + i>)2 + {l-a){u- v)'^' 
x = g{a,u,v) = ^ (4.1.2) 



V 

where a = a{a) = cos^ ^ = (1 + cosa)/2. In particular, for u 



ex 
g{a, u, u) = u{l — v^) = ^(1 — cos — ). (4.1.3) 

The function g{a^ w, v) has the foUowing properties (we shaU not use them further 
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Proposition 4.1.3. u > g{a,u,v) > Q, v > g{a,u,v) > 0, g{a^u^v)'^ > 0, 
g{a,u,v)^>Q, [^ — ^\' ' j >0, \^ ^\' ' J > 0. 

Proof. Suppose that u > v. The expression {u — x){v — x) equals uv > auv if x = 
and {u — x){v — x) equals if x = -u. Thus the smallest solution g{a, u, v) of the 
equation {u — x){y — x) — auv satisfies v >u> g{a, u, v). 

We have 2g{a, u, v)'^ = l — {a{u+v) + {l—a){u—v))/^ya{u + v)'^ + {1 — a){u — v)^ 
> if {a{u + v) + {1 - a){u - v)) / ^y a{u + v)"^ + {1 -a){u-v)'^ < 1. Equiv- 
alently, for —1 < t = {u — v)/{u + v) < 1 we should prove that (a + (1 — 
a)t)/y/a+ (1 -a)t2 < 1. If -1 < t < 0, this is obvious. For < t < 1, we have 

(a + (1 - a)t2)i (^(a + (1 - a)t) / y/ a + {1 - a)t^y = (1 - a)(a + (1 - a)t^) - (1 - 

a)t{a + {l-a)t) = (l-a)a(l-t) > 0. It follows that {a + {l-a)t)/^ya + {1 - a)t'^ < 
(a + (1 - a))/^a + {l-a) = 1. 

By definition, we have {u — g^a, u, v))/u = av/{v — g{a, u, v). It follows that 
((tt — (/(a, u, v))/u)^ > because (/(a, u, v)'^ > 0. 

It finishes the proof. 

Theorem 4.1.4 (about the narrow place of type (I)). For any elliptic convex 
polygon M. on a hyperbolic plane there exist its four consecutive vertices Aq, Ai, 
A2 and As (where Aq = A3 if A4 is a triangle) such that for orthogonal vectors di , 
82 and ds to lines (AqAi), {A1A2) and {A2A3) respectively directed outwards of Ai 
and with df = 62 = S^ = —2 one has (5i,52) = 2cosai, (52,^3) = 2 cos 02 and 

either (61,63) < 2 or {61,63) < 4{cos^ + cos^f - 2 < U (4.1.4) 

where ai = A0A1A2 and a2 = A1A2A3. 

Moreover, the Gram graph of {61, 62, 63} is not connected (i.e. this set is union 
of two non-empty orthogonal subsets) if and only i/cti = 02 = f • 

Proof. To prove Theorem, we take a point O inside of TW = AiA2...An. Let Bn 
and Bi2 are terminals at infinity of the line h = (Ai-iAi) where Bn, Ai-i, Ai and 
Bi2 are four consecutive points of the line. We introduce angles ai = Ai-iAiAi+i, 
9i = BiiOBi2 and 6'(i+i)i = B(^i+i)iOBi2. 
By Lemma 4.1.2, 

sm o sm -^ ^ ' 



2 cos ai = 4 ^— 5 ^ 2. 

sin f sin ^ 

Equivalently, 

sin If sin ^'+' "" 



2 """ 2 



gjj^ 9i-9(^+i)i g.^ ^i+i-^(i+i)» 1 + cos ai 



(4.1.5) 



By Lemma 4.1.1, we get 



(^i - ^(i+i)i)(^i+i - ^(i+i)i) 1 + cos 



ai 2 o^t 



<— i = cos^^. (4.1.6) 



OiOi+i 2 2 

It follows that 



„/'„, n n \ r A -I ^\ 
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(see (4.1.2)). 

To prove Theorem 4.1.4, we choose a hne k with the minimal angle 9i. Let this 
line be l2- Thus 

^2 = min^,. (4.1.8) 

i 

It follows that 6*1 > 6*2 and 6*3 > 6*2. We shall then prove the inequalities (4.1.4). 

If lines /i, I3 intersect, we have (5i, ^3) < 2 and (4.1.4) is valid. Suppose that the 
lines 1 1 and I3 do not intersect. 

There exists a line ![ = {B'iiB'12) with terminals B'n and B'12 at infinity such 
that the line ![ is contained in Ti^g , points -Bii, B'n and -B12, -612 are contained in 
the same half-planes bounded by I2, the line ![ has the same angle cti = B[iA[B22 
(as /i) with the line I2 (we denote by A[ their intersection point), and the angle 
B[^OB[2 of I'l is equal to 6*2. We denote by 5^ the orthogonal vector to I'l directed 
outwards of A4 and with (5^)^ = —2, and we denote 6*21 = B2iOB[2. 

Similarly, there exists a line l'^ = (S31S32) with terminals B'^i and S32 at in- 
finity such that the line /g is contained in Ti-^g , points S31, Sgj^ and S32, -B32 
are contained in the same half-planes bounded by I2, the line l'^ has the same angle 
«2 = -B21A2-B32 (as /3) with the line I2 (we denote by A2 their point of intersection) , 
and the angle B'^^OB'^2 of ^3 is equal to 6*2. We denote by ^3 the orthogonal vector 
to /g directed outwards of A4 and with (^3)^ = —2, and we denote 6*32 = B'^iOB22- 

Since the lines h and /3 do not intersect, by our construction, any interval with 
terminals at ![ and l'^ intersects both lines h and /3. It follows that distance be- 
tween lines I'l and l'^ is greater than distance between lines /i and l^. It follows 
{^ij S3) < {S'li S's)- It is sufficient to prove (4.1.4) for lines l[ and /g. (These geomet- 
rical considerations are related with properties in Proposition 4.1.4 of the function 
g{a,u,v).) 

By Lemmas 4.1.1, 4.1.2, and (4.1.7), (4.1.3), 

02~\~02 — P91 — Ono • 02~\~02 — ^91 — 0^^ 

sm — — o — — sm ^ ^ 21 32 



{Si, Ss) < {S[, 6',) = 4 V ,. ■ ,. '- 2 < 



sm ^ sin -^ 



^ {62 + O2 - 6*21 - 1932) (6*2 + O2 - ^21 ~ ^32) _ 2 < 
^2^2 

^ (^2 + ^2 - ^(«l, 02, 62) - ^(«2, ^2, ^2))(^2 + ^2 - gjai, 02, 62) - ^(^2, ^2, ^2)) ^ _ 

^2^2 

4(v^ + v^)' - 2 = 4(cos ^ + cos ^f - 2 

where a^ = cos^^. It proves (4.1.4). 

Elements 61,62,63 generate the hyperbolic 3-dimensional vector space defining 
the hyperbolic plane. Otherwise their lines either have a common point or are 
orthogonal to one line which is not the case. If the Gram graph of these elements 
is not connected, two of these elements generate a 2-dimensional hyperbolic vector 
subspace. Elements 61 and 62 cannot generate a hyperbolic 2-dimensional vector 
subspace because their orthogonal lines have a common point R_|__|_/i where h^ > 
and /i 7^ 0. Then (h, 61) = {h, 62) = 0. The same is valid for 62 and ^3. Thus the 
Gram graph of {61, 62, 63} is not connected if and only if {61, 62) = (^3,^2) = 0. 
Equivalently, cti = 02 = §• It finishes the proof of Theorem AAA. 
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Theorem 4.1.5 (about narrow places of types (II) and (III)). For any 

elliptic convex polygon M. having more than 3 vertices (i.e. it is different from a 
triangle) on a hyperbolic plane, one of two possibilities (11) or (III) below is valid: 
(II) There exist its five consecutive vertices Aq, Ai, A2, A3 and A4 (where 
Aq — A4 if JH is a quadrangle) such that for orthogonal vectors Si, 62, S3 and 64 
to lines (AqAi), {A1A2), (A2A3) and (A3A4) respectively directed outwards of M. 
and with df = 62 = S^ = d1 = —2, one has (5^, 5i+i) = 2 cosai, z = 1, 2, 3, and 

either (61,63) < 2 or {61,63) < 4{cos^ + cos^f - 2 < U, (4.1.9) 

{61,64) < 



4 max 

0<t<l 



(^/al + {l-al)t^ + y/a2 + (1 - 02)^2 + ^03 + a3t + t'^/4 



1 + t 



( 



4 max 



V 



tti a2 , ^3 

cos h COS h COS — , , 

2 2 2 ' ' 



2 + ,/2cos2^ + i 



4 



/ 



-2 



2 <34 
(4.1.10) 



where ai = Ai^iAiAi^i, i = 1, 2, 3, and a^ — cos ^. Moreover, the set {61, 62, 
63, 64} has a connected Gram graph. 

(III). There exist its six consecutive vertices Aq, Ai, A2, A3, A4 and A^ (where 
Aq — A^ if M. is a pentagon) such that for orthogonal vectors 61, 62, 63, 64 and 65 to 
lines {AqAi), {A1A2), {A2A3), {A3A4) and {A4A5) respectively directed outwards 

61 = 6^ — —2 one has {6i,6i+i) — 2cosQ;i, 



of M and with 6l = 62 
i = 1,2, 3, 4, and 



Si 



and 



either {61,63) < 2 or (5i, ^3) < 4(cos ^ + cos ^)2 - 2 < 14, 
either {63,65) < 2 or (^3, ^5) < 4(cos — + cos — )2 - 2 < 14, 



{61,65) < 



(4.1.11) 
(4.1.12) 



4 max ( ( \/ai + (1 - ai)s'^ + J a2 + (1 - 02)5^+ 
o<t<s<i LVV ^ ^ / V \ 



+ Wa3 + a3(.-t) + ^^^^i^+ (^-"^l(^ + ^^V V a4 + (l-a4)t2 



[s-t)'' 



/((l + .)(l + t)) 



-2 = 



4 max 



Ctl (X2 (y.3 04" 

COS V COS V COS V COS — , , 

2 2 2 2 ' ' 



(2 + y/2 cos2 ^ + i + cos 



2 < 62 



(4.1.13) 
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where ai = Ai^iAiAi^i, i = 1, 2, 3, 4, and ai = cos^ ^. Moreover, the set 
{5i, ^2, ^3, ^4, 65} has a connected Gram graph. 

Proof. Like for the proof of Theorem 4.1.4, we take a point O inside of Ai and 
introduce angles 9i, 6'(j_|_i)j. We choose such a consecutive numeration of vertices 
of M that 

92 + 9s = min {9, + 9,+i) and 92 < 9s. (4.1.14) 

i 

Then 6'i > 6*3 > 6*2 and, hke for the proof of Theorem 4.1.4, we get (4.1.9) and 
(4.1.11) (see considerations below about cases (II) and (III)). 

Below we consider two cases: 

Case (ii): ^4 > (^2 + ^3)/2. We then prove (4.1.10). 

For 1 > ai, a2, 03 > and < t we introduce a function 

/p2(ai, a2, 03, t) = 

(y^ai + (1 - ai)t2 + y/a2 + (1 - a2)t^ + ^a^ + a^t + f^/A) -^ 

= ^ ^ . (4.1.15) 

1 + t ^ ' 



We prove that 



(^1,54) < 4 max /p2(ai, ^2, as, ^) - 2. (4.1.16) 

o<t<i ^ 



If (5i, 54) < 2, it is true because 4/p2(ai, 02, 03, 1) - 2 = 2((2 + ^2as + 1/4)^ - 
1/4) - 2 > 2((2 + 1/2)2 _ 11^^ - 2 = 10. 

Like for the proof of Theorem 4.1.4, we can suppose that 9i = 9^ (instead of 
9i > 93) and 94 = {92+9s)/2 (instead of ^4 > (^2+^3)/2). We denote c = (^2+^3)/2 
and z = {9s- ^2)/2 > 0. Then ^2 = c - ^ and ^3 = c + 2. 

Like for the proof of Theorem 4.1.4, we have 

((5i,54) + 2)/4< 
{9s + 92 + 9s — 6*21 — 6*32 — 6*43) (c + 6*2 + 6*3 — 921 — 9s2 — ^43) 



< 



9sc 

{9s + 92 + 9s- g{ai, 9s, ^2) - ^(^2, ^2, ^3) - ^("3, ^3, c)) x 
(c + 6'2 + 6'3 - g{ai, 9s, 6*2) - fi'(a2, 92, 9s) - g{as, 93, c))/9sc = 



{z/2 + Vaic2 + (1 - ai)z2 + Va2c2 + (1 - 02)2^ + ^/ _ + ascz + asc'^)x 



z^ 



{-z/2 + y'aic'^ + (1 - ai)z'^ + y'a2c'^ + (1 - a2)z'^ + \l -j- + o-scz + asc'^)/{c+z)c 



z^ 



( v/^I^2-T(T^^^0^ + v/^^^^TTT^^^^ + Y^f + a3cz + a3c2)2 - ^2/4 

(c + z)c 

( ^oi + (1 - ai)t2 + ^a2 + (1 - a2)t2 + y^f + 03^ + 03 j -^2/4 
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If one of angles cti or a2 is not 7r/2, Gram graph of {5i, ^2, ^3} is connected. 
Then Gram graph of {5i, ^2, ^3, ^4} is connected because 5i, ^2, ^3 generate the fuU 
3-dimensional hyperbohc vector space. If a\ = ai = t^ /'2. (equivalently, (5i, ^2) = 
(^3, ^2) =0), then 5i, ^3 generate a hyperbohc 2-dimensional subspace and (5i, ^3) 
7^ 0. If Gram graph of {5i, ^2, ^3,54} is not connected, (5i, ^4) = (^3, ^4) = 0. It 
follows that ^4 = A52, A G M. Then sides of M. orthogonal to 82 and ^4 should 
coincide. We get a contradiction. 

Case (ii) gives polygons M. which satisfy the condition (II) of Theorem 4.1.5. 
We mention that by Theorem 4.1.4, any quadrangle M. satisfies the case (II) of 
Theorem 4.1.5. Really, for a quadrangle, (^1,54) < 2. 

Case (iii): 6*4 < (6*2 + 0^)/2 and M. has more than 4 vertices. We denote 

(^2 + ^3)/2 = C. 

By (4.1.14), Oi > ^3, ^4 + ^5 > {O2 + 93) = 2c. It follows, 65 > c. 
Since ^3 > c, ^4 < c and ^5 > c, like in the proof of Theorem 4.1.4, we get the 
inequality (4.1.12). 

For 1 > ai, a2, 03, 04 > and < s,t, we introduce a function 

/p3(ai, a2, a3,a4, s, t) = 
( ( A/aTTO^^-oaJs^ + A/a2 + (1 - 02)5^+ 



We prove that 

(5i, 55) < 4 max fpsiai, 02, 03, 04, s, t) - 2. (4.1.18) 

0<t<s<l 

If (5i, ^5) < 2, it is true because /p3(ai, 02, 03, 04, 1, 1) = 4. If (5i, ^5) > 2, like 
for the proof of Theorem 4.1.4, we can assume that 9i = 6*3 (instead of 9i > 6*3) 
and 6*5 = 2c — 6*4 (instead of 6*5 > 2c — 6*4). 

We denote c = {62 + ^3)/2, z = (63 - ^2)/2 and zw = (^5 - 04)/2. We have 
Oi = 9s = c + z, 02 = c — z, 04 = c — w, 65 = c + w. By definition, c > 0, z > 0, 
w > 0. Moreover, z > w because 6*3 + 6*4 = 2c + ^ — ty > 6*2 + 6*3 = 2c (we use 
(4.1.14)). 

Like for the proof ot Theorem 4.1.4, using Lemmas 4.1.1, 4.1.2, we get 

((5i,55)+2)/4< 

(6>3 + 6>2 + 6>3 + 6>4 — 6>21 — 6>32 — 6*43 — 6*54) (6*5 + 6>2 + 6*3 + 6*4 — 6>21 — 6*32 — 6*43 — 6*54) 

^3^5 

(6*3 + 6*2 + 6*3 + 6*4 - ^(«i, 03, 02) - 9{a2, 02, 03) - gias, 03, 04) - 9{a4, 04, 05))^ 
IQ t Q t a I z) „/'„ a n \ „i ^. a n \ „i „. a a \ „i „. a n \\ ita n \ 
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((Vaic2 + (l-ai)22 + v/a2c2 + (1 - a2)z^+ 
asc^ + asc{z - w) -\ : \ : \ 



4 ' 4 

a4c2 + (1 - a^)w^f - ^^-"^^ \ /((c + z){c + w)). 



Denoting s = z/c and t = w/c, we get 

(((^1, (^5) + 2)/4 < /p3(ai, a2, as, 04, s, t) 

where < t < s < 1. 

Similarly to the case (ii), one can easily prove that Gram graph of {5i, S2, ^3, ^4, 
^5} is connected. Thus, for the case (iii), we get the case (III) of Theorem. 

To finish the proof of Theorem, in Lemmas 4.1.6 and 4.1.7 below, we find 
maxo<t<i fp2{ai, ^2, as, t) and maxo<t<s<i fpsiai, a2, as, a4, s, t). 

Lemma 4.1.6. For < ai, a2, as < 1 and t > the function 

/p2(ai, a2, as, t) = 

(y/ai + {l-ai)t^ + ^/a2 + (1 - a2)t2 + ^^as + ast + t^/4) - ^ 



1 + t 



has the maximum 



max /p2(ai, a2, as, t) = max[/p2(ai, a2, as, 0), /p2(ai, a2, as, 1) 



max 



2 (2 + J2as + i 



Lemma 4.1.7. For < ai, a2, as, a4 < 1 and s, t > the function 

/ps(ai, a2, as,a4, s, t) = 



( ( v^aT-TO^^-o^ys^ + V'^2 + (1 - a2)s2+ 

+ Jas + as{s-t) + ^^^^^+^-^-^^4^±^+Va^ 



{s-ty 



/((l + .)(l + t)) 



has the maximum 

max /ps(ai, a2, as, a4, s, t) = 

0<t<s<l 

max[/ps(ai, a2, as, a4, 0, 0), /ps(ai, a2, as, a4, 1, 0), /ps(ai, a2, as, a4, 1, 1)] = 



max 



{y/ai + y/a2 + \/as + ^/cH 



2 

(2 + y/2as + i + a4) -i 
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Proof of Lemma 4. 1.6 for 1 > ai, as > 0.2723, and 1 > ag > 5/32 = 0.15625. We 
shall use Theorem 4.1.5 only for 1 > a^ > 1/2 when a polygon Ai has acute angles. 
Therefore, we restrict proving Lemma 4.1.6 for the parameters ai above. 

To prove Lemma 4.1.6 for parameters ai, 02, 03, it is sufficient to show that 

/p2(ai, a2, as, t)'lt > 0, if < t < 1. (4.1.19) 

For 1 > a > and < t < 1 we consider 

.= ^±ii^^. (4.1.20) 

1+t ^ ' 

We have 

, {l-a)t^ + 2{l-a)t-a ^ 1 „ 2 ,, 101^ 

-u^ = TL 775 = 1 — a — 7- -77, Uff = — -^. (4.1.21 



It follows, 



(^^);; = ^-M-^K)' + ^«< 



a+ (l-a)t'= 

'X 



t + 1 

-(1/4)(1 - a)^t4 - (1 - aft"^ + (3/2)(a - a')^ + (a - a^)t + a - a74 

(l + t)4 



> 



.2X-I 



a + (l -a)r 

'X 



t + 1 
(1/4)(1 - 0)2^2 - (1 - aft^ + (3/2)(a - a'')^ + (a - a^)^ + a - a74 



(1 + ^) 



4 



a+(l-a)t2^^ ^ (l-a)(^a- f)t2 + (a-a2)t + a-ia2 



a-\a^, ifi<a<l 



t + 1 ; (1+t) 

3 {a -^a , 11 2 



(^±iL_^y'x(l+t)-^x a-\a\ ifO<a<i&0<t<,^ 

[-| + 6a-4a2, if < a < i & ^^ < t < 1. 
It follows. 



3 I a — ^a", if -i < a < 1 

,2\-i I 4 ' 3 - - 



1^2 

11 3 - « - -L 

4a 



^±iL_^] \(l + t)-4x<j a-ia2, ifO<a<i&0<t<,_,,„ 

-| + 6a-4a2, if < a < i & ^^ < t < 1 



— - 
/a+(l — a)t \ ^ ,195 , 9x ,/ Nzi /. N 
^ — —^\ xmin a- -a2,-- + 6a-4a2 / 1 + t^. 4.1.22 
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Here min (a — |a^, — | + 6a — 4a^) > if a > 1/4. If follows 
, 1,,, min(a — ^a^, — I + 6a — 4a^) 

-u^ ;; > — ^ — ^ ^ . > 4.1.23 

if 1 > a > i and < t < 1. 

For 1 > a > and < t < 1 we consider 

t^/4 + at + a 

V = ; . 

1 + t 

We have 



(4.1.24) 



t74 + t/2 _l 1__ _ 1 

""'- (l + t)2 "4 4(l + t)2' ''«-2(l + t)3- ^^-^-^^^ 



It follows 



3 



, 1,,, /t74 + at + a\ 2 -(1/64)^4 -(l/16)t^ + (l/4)at + (l/4)a 

^''^- = [ 1 + t ) " (TTW' ^ 

t^/4: + at + a\~^ -(l/64)t^ - (1/I6)t3 + (l/4)at + (l/4)a _ 
1 + t ) "" (T+tj^ ~ 

tV4 + at + aY^ -(5/64)^3 + (l/4)at + (l/4)a 
1 + t ; ^ (l + t)4 - 

_3 f i*^' if ^ < a < 1 

ty^ + at + ay-^ ^\ , if0<a<^&0<t<4./f 



-^ + ia, if0<a<^&4/f <t<l. 



It follows 



_3 C ^a, if ^ < a < 1 

/ t74 + at + a \ \(i^^)-4^ ) 1^^ if0<a<^&0<t<4^ 



^ + ia, ifO<a< ^ &4^<t<l 



> ^ /-t-ruL-ru ^ ...... y^u,, 64 ^ 2"-^ /^ ^ 26) 



't2/4 + at + a\ ^^ min(ia, -^ + ia) 
1 + t J ^ (1 + t) 

Here min(|a, — ^ + |a) > if a > j^. If follows 



. ix„ . , X 3 min(ia, — to- + ^a) 

(^^);; > (1/4 + 2a)-^ X "-^ f\ ' ^ (4.1.27) 

(1+t) 2 



if 1 > a > 5/32 = 0.15625, < t < 1. 
We have 



/tV4y_i 1 /tV4Y 



<->/-! I J.N'? ■ 



(4.1.28) 
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We denote for 1 > a^ > and < t < 1, 

ai + {l-ai)t^ a2 + (l-a2)t^ t^/4 + a3t + as 

Ul = , U2 = "Us = 

1+t ' 1+t 1 + t 

and 



We have 

(«;2);; = (yj'^f + 2ww';^ > 2ww',[. (4.1.29) 

From (4.1.23), (4.1.27), (4.1.28) and (4.1.29), for 1 > ai, a2, > 1/4 and 1 > ag > 

5/32, we get for fp2 = w^ - (l/4)t7(l + t), 

/p2(ai, a2, as, t)"^ > 



2 X 



2 ({ai + (1 - ai)t2)5 + (a2 + (1 - 02)^^)^ + {t'^/A + a^t + ag)^ 
min (ai — -a^, — t + 6ai — 4a^) + min (a2 — -02, —7 + 6a2 — 4a2) + 



1 Fi "1 \ "1 

(l/4 + 2a3)-2min(-a3, -^ + 2''3)J-2 



[2 (v^ + y/a^ + y/a^ X 



/{l + tf> 



min (ai a?, h Gai — 4a?) + min (a2 a,, h 6a2 — 4a9) + 

^44 ^44 



/, , . „ x_3 . ,1 5 1 n\ 1 

(l/4 + 2a3) 2mm(-a3, -— + -a3)j -- 



/(l+t)^>0 (4.1.30) 



if 



I min (ai — -a^, — t + 6ai — 4a^) + min (a2 — -a|, — t + 6a2 — 4a2) + 
(l/4 + 2a3)-'min(Ja3, "^ + ^«3)) > ^. (4.1.31) 

Thus, (4.1.19) is true if 1 > ai, a2 > 1/4, 1 > as > 5/32 and (4.1.31) is vahd. For 
1 > ai, a2 > a > 1/4 and arbirary 1 > as > 5/32, (4.1.31) is vahd if 

2(2v^+yj)(-^+6a-4a2)>i. (4.1.32) 

It is true if 1 > a > 0.2723. 

It proves Lemma 4.1.6 for 1 > ai, a2 > 0.2723 and 1 > as > 5/32 = 0.15625. 

Proof of Lemma 4-1-7 for 1 > ai, a2, as, a4 > 0.37646. We shall use Theorem 4.1.5 
only for 1 > a^ > 1/2 when a polygon Ai has acute angles. Therefore, we restrict 



,„ /I 1 T C J-U„ „„„„.„„J- „ „ „1 ,.„ 
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To prove Lemma 4.1.7 for parameters ai, 02, 03, 04, it is sufficient to show that 

/p3(ai, a2, as, 04, s, t)^'^ > 0, if < t < s < 1, (4.1.33) 

/p3(ai, a2, as, 04, s, 0)';, > 0, if < s < 1; (4.1.34) 

and 

/p3(ai, a2, as, 04, s, s)'^ > 0, if < s < 1. (4.1.35) 

We have 

/p3(ai, «2, as, ^4, s, 0) = 

( a/gi + (1 - ai)s2 + A/a2 + (1 - a2)s2 + Was + ass + ^ + -Jal j -^ 

m ■ 

(4.1.36) 
Like for the proof of Lemma 4.1.6, it follows for 1 > ai, 02 > ^, 1 > as > ^, 
1 > a4 > that 

/p3(ai, ^2, as, ^4, s, 0)^'^ > 

2 ('(ai + (l-ai)s2)^ + (a2 + (l-a2)s^)^ + (sV4 + ass + 03)^ +01) x 

15 15 

min (ai — -a^, — -: + 6ai — 4a^) + min [0,2 — -02, ^t + 602 — 4a2) + 



/1 /. ^ \-3 . /I 5 1 ,\ 1 

(l/4 + 2a3) 2mm(-a3, -— + -a3) 1 -- 



[2 (v/qT + v^ + v^ + v^) X 



/(! + «)'> 



I min (tti — -af , — -: + 6ai — 4a^) + min (a2 — -02, — - + 6a2 — 4a2) + 



if 



x-3 .1 5 1 A 1 

(l/4 + 2a3) 2niin(-a3, -— + -a3)j -- 



(v^ + V^ + V^ + Vtt4) X 



/(l + s)3>0 (4.1.37) 



min (tti — -af , — -: + 6ai — 4a^) + min [0,2 — -02, —7 + 6a2 — 4a2) + 

(l/4 + 2a3)-^min(Ja3, -^ + ^«3)) >\. (4.1.38) 

Thus, (4.1.34) is true if 1 > ai, 02 > 1/4, 1 > 03 > 5/32, 1 > 04 > and (4.1.38) 
is valid. 

For 1 > ai, a2 > a > 1/4, arbirary 1 > 03 > 5/32 and arbitrary 1 > 04 > 0, we 
have (4.1.38) if (4.1.32) is vahd. It takes place if 1 > a > 0.2723. 

We have 
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ai + (l-ai)s2 + ^Ja2 + (1-02)^2 + ^^03 + (l-as)^^ + ^J a^ + (1-04)^2 



For 1 > a > and < s < 1 we denote 



(4.1.39) 



e = "+^^~"^"' (4 1 40) 







-2a + 2s(l-a) 2s 2a 




(1 + S)3 (1 + S)3 (1 + S)2' 






2 + 4a + (4a-4)s 2 - 4s ^ 4a 
(l + s)4 (l + s)-* ' (l + s)3 


It follows, 







(4.1.41) 



_3 (-2a2 + 4a - 2)s^ + (-Sa^ + Sa)*^ + (a^ + a) 

e 2 X ^ -^ ^ > 

(l + s)6 

_3 (-2a^ +4a- 2)s^ + (-Sa^ + 3a)s^ + (a^ + a) 

e 2 X ^ — ^ — ^ = 

(l + s)6 



_3 (-Sa^ + 7a - 2)s2 + (a^ + a) 

e 2 X ^ r. — ^T7^ — ^ > 

(l + s)6 

3 ( a'^ + a, if 1 > a > I 

-4a2 + 8a - 2, if | > a > 



min(a^ + a, — 4a^ + 8a — 2) 



e-2 X ^ — ^ —^ '-. (4.1.42) 

(l + s)6 ^ ^ 

Here min(a2 + a, -4a2 + 8a - 2) > if a > (2 - V2)/2 = 0.292893... . It follows 
min(a^ + a, — 4a^ + 8a — 2) 



(e^)L > ' ^ :; " ^ ^ ^ > if 1 > a > (2 - v^)/2. (4.1.43) 



For 1 > ai, a2, as, a4 > we denote 

_ a^ + (1 - ai)s'^ 
""'' (l + s)2 • 

We get 

/p3(«l, ^2, as, «4, S, S)ss > 

1111 



(4.1.44) 
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2 ( A/ai + (l-ai)s2 + A/a2 + (l-a2)s2 + A/a3 + (1 -as)^^ + A/a4 + (l -04)^2 j x 

((min(a^ + ai, — 4a^ + 8ai — 2) + min(a2 + 02, — 4a2 + 8a2 — 2) + 

+ min(a3 + as, -Aaj + Sa^ - 2) + min(a4 + 04, -4al + 8a4 - 2)) /(I + s)'^ > 

((min(a^ + ai, —Aaf + 8ai — 2) + min(a2 + 02, — 4a2 + 8a2 — 2) + 

+ min(a3 + as, -4a3 + 803 - 2) + min(a4 + 04, -4a4 + 804 - 2)) /(I + s)"^ > 

(4.1.45) 

if 1 > Oi > (2 - v^)/2 = 0.292893... . 

Let us prove (4.1.33) for 1 > ai, a2, as, a^ > 0.37235. We denote 



_ a + a{s-t ) + ^^^ + ^^-"y+^)' _a + as + ^+ t{-a - as + f ) + ^ 



t+1 t+1 

We have 

, _ -2a - 2ag + f - ^ + I + ^ _ 1 _ (1 - g)^ + 8a(g + 1) 
■''~ (t + l)2 ~4 4(t + l)2 

(l-s)2 + 8a(s + l) 



// 



•^" 2(t + 1)3 

For 1 > a > and 0<t<s<l, we have 



{fhu = r'^{-l{ftf + lffti)- 



r i (--t^ - -t^ + ( A,2 + (3 _ ^ 3 2 

■^ ^ 64 16 ^32 M 16^ 4 ^ 

-1-1 t^ Q T "1 

^2 M 16^ 4 64^ 

((-^a + g)«' + (-2a2 + -a - — )s2 + (-4a2 + -a)s + {-2a^ + -a))t+ 



.g4" , ,2- — )s3 + (a2 + -a)s2 + (2a2 + -a)s + (a2 + -a))/(t + l 
For 1 > a > I and 0<t<s<lwe have 



( — s + (-a )s + -a )t + 

^2 M 16^ 4 64^ 

((--a + -)s^ + (-2a^ + 7a - 777)5^ + (-4a^ + 7^)5 + {-2a^ + -a))t+ 
^^4 8^^ 4 16^ ^ 4 ^ ^ 4 ^^ 



f — .s"^ + (-a l.s^ + (n^ 4- -a]s^ + ('2a2 + -al.s + fa^ + -a)) > 
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((-^a + g)«' + (-2a' + 4^ - -^)s^ + (-4a' + ^a)s + (-2a' + -a))t+ 



min f ^s^ + (^a - ^)s3 + (a' + ^a)s' + (2a' + ^a)s + (a' + ^a), 

(--a + — )s^ + (-2a' + -a - -)s^ + (-3a' + -a)s' + -as + (a' + -a) 
W 64^ ^ 4 4^^ 4^2^ 4^^^^_4g^ 

Here ^s'* + (ia— ^)s^ + (a^ + ia)s^ + (2a^ + ia)s + (a^ + ia) > a^ + ^a. Let us prove 
that (-ia+ii)s4 + (-2a2 + |a-i)s3 + (-3a2 + |a)s2 + ias + (a2 + ia) > ga^ + ia. 
Equivalently, we should prove that ( — |a + ^)s'* + (— 2a^ + ^a — j)s^ + (— 3a^ + 
fa)s2 + las+fa2>0. 

Assume that 1 > a > j^. Then -Ja+JJ < and (-ia+ 1^)^"^ + (-2a2 + Ja - 
i)s3 + (-3a2 + |a)s2 + ias + ga2 > r{a,s) = {-2a'^ + ^a--§^)s^ + {-3a^ + la)s^ + 
ias + ffa^. We have r(a, 0) > and r{a, 1) = -^a^ + x^ ~ M - 0- ^^ follows 
that r{a, s) > if i<^ = 3(-2a2 + |a - ^)s + (-3a2 + |a) < 0. The last is valid if 
i<,(a, 0) = -3a2 + f a < and K,(a, 1) = 3(-2a2 + |a - ^) + (-3a2 + |a) < 0. 
It is true if 1 > a > | when — 3a^ + fa < 0. Suppose that | > a > j^. Then 
-3a2 + fa > 0. If also -2a2 + |a - ^ > 0, then r{a, s) > 0. If -2a2 + |a - ^ < 0, 
we have r{a, s) > {{-2a'^ + f « - m) + (-3a' + ja))s^ + ^as + ga^. If {-2a^ + 
|a — ^) + (— 3a^ + |a) < 0, we again have that r{aj s) > min(r(a, 0),r(a, 1)) > 0. 
If (-2a^ + 2 "^ ~ ^) + (-3a^ + |a) > 0, we obviously have r{a, s) > 0. 

Assume that g > a > 1/4. Then {-\a + ^)s^ + {-2a^ + Ja - ^s^ + (-3a2 + 
|a)s2 + ias + ga^ > r{a, s) = {-2a'^ + |a - l)s^ + {-3a^ + |a)s2 + ^as + ffa^. 
Arguing with the function r{a, s) like above, we prove that r{a, s) > 0. (Here one 
can get even better result.) 

Thus, for 1 > a > ^ and < t < s < 1, we have 

ifhn > 

ii«2 I 1 „ 



We have / < (1 + (5/4)a)/(t + 1). It follows 



119 1 

if-'Yu > (1 + 7a)"^ X M^±4^ (4.1.48) 

v^ ytt - V 4 y (t + l)i 

if 1 > a > i and 1 > s > t > 0. 

We have (t - sf/{t + 1) = t + 1 - 2(s + 1) + (s + l)7(t + 1), it follows 



jt-s)^ \" ^ 2{s + l) 



< 7-^T7^^T7- (4.1.49) 
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Like for the proof of Lemma 4.1.6, from (4.1.23) and (4.1.48), it follows that for 
1 > ai, a2 > 0, 1 > as, 04 > i, 

/p3(ai, a2, as, 04, s, t)^^ > 
(l + s)-^(l+t)-3 [2 (^Vai + (1 - ai)s2 + v^a2 + (1 - 02)5^+ 



/, 5 x_3,ll 2 1 N . / 1 2 5 ^ . 2n\ (■5 + 1)^ 

(1 + 7^3) "" (^ttg + -as) + mm (04 - -a^, -- + 604 - 4a4^ 



4 -/ vg4 o 4 ^/ V - 4 -i' 4 -i/ , 2 



> 



(l + s)-^(l+t)-3[2(v/^+v^+v^+v^) X 



5 ,_3 11 2 1 \ 1 2 ^ 2\\ 

(l + -a3) 2(--a3 + -a3)+mm(a4- -04, -- + 604 - 404) - 2J. 

4 64 4 4 4 J ^4-^5Q^ 

Thus, /p3(ai, a2, 03, 04, s, t)"^ > if 1 > ai, a2 > 0, 1 > 03, 04 > ^ and 

(v^ + V^ + V^ + v^) X 

(1 + -a3)"^(— a3 + -as) +min(a4- -ai -- +6a4-4a|) ) > 1. 

4 64 4 4 4 ; (4^5^^ 

For arbitrary 1 > ai, a2, as, a4 > a > | it is valid if 

^ /, 5 , 3,11 o 1 , 1 oX 1 

v^(^(l + -a)-^(-a2 + -a)+a--a2j > -. (4.1.52) 

It is true if a > 0.37646. 

It proves Lemma 4.1.7 for 1 > ai, a2, as, a4 > 0.37646 when all three conditions 
(4.1.33), (4.1.34) and (4.1.35) are valid. 

Using Theorems 4.1.4 and 4.1.5, we can divide all convex elliptic polygons on 
the hyperbolic plane in three types. This subdivision is very useful for fundamental 
polygons of finite volume of reflection groups, and we shall use it in Sects. 4.3 and 
5. 

Theorem 4.1.8. Let M. he an elliptic convex polygon on a hyperbolic plane. Then 
M. has one of types (I), (11) or (III) (or the type (I), (II) or (III) of its narrow 
place) below: 

Type (I): There exist its four consecutive vertices Aq, Ai, A2, A3 (where Aq = 
A3 if Ai is a triangle) with angles ai = A0A1A2, 02 = A1A2AS 7^ ^ and such that 
for orthogonal vectors di, 62 andds to lines (AqAi), {A1A2) and {A2AS) respectively 
directed outwards of M. and with 5^ = 5| = 5| = —2 one has (5i,52) = 2cosai, 
('^2, 5s) = 2 cos a2 7^ and 

either fr^, . r^ol < 2 or (f,. . ^o^ < 4('cos — + cos — ^^ _2<TA ('4.1 .531 
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The set {5i, ^2, ^3} generates the 3-dimensional hyperbolic vector space and has a 
connected Gram graph. Any triangle or quadrangle M. has the type (I). 

Type (II): There exist its five consecutive vertices Aq, Ai, A2, A^, A4 (where 
Aq = A4 if Ai is a quadrangle) with angles ai = A0A1A2 = f , a2 = A1A2AS — ^, 
ct3 — A2A^A4^ and orthogonal vectors Si, 62, Ss and ^4 to lines (AqAi), {A1A2), 
(^2^3) and (A3A4) respectively directed outwards of M. and with 51 — 82 — 5l — 
6l = —2 such that {Si, 62) = {S2, Ss) = 0, (^3, ^4) = 2cosas, 



(5i, Ss) < 6 (4.1.54) 



and 



(5i, ^4) < 4 max 



(v^+cos^ 

V 



2 (2 + ,/2cos2f + i -i 



2 



2 < 



< 10 + 8^2 = 21.313708... . (4.1.55) 

Moreover, the set {61,63,64} generates the 3- dimensional hyperbolic vector space 
and has a connected Gram graph. 

Type (III). There exist its six consecutive vertices Aq, Ai, A2, A^, A4, A^ 
(where Aq = A^ if A4 is a pentagon) with right angles ai = AqAiA2 = f , 0^2 = 
A1A2A2, = f , 0^3 = A2A3A4 = ^, a^ = A^A^A^ = ^ such that for orthogonal 
vectors 61, 62, 63, 64 and 65 to lines (AqAi), {A1A2), (A2A3), (A3A4) and {A4A5) 
respectively directed outwards of M. and with 61 = 62 = 6^ = 6l = 6^ = —2 one has 
{61, 62) = {62, 63) = {63, 64) = {64, 65) = and 

(61,63) < 6, (4.1.56) 

(53,55) < 6, (4.1.57) 

and 

(61,65) < 30. (4.1.58) 

Moreover, the set {61, 63, 6^} generates the 3-dimensional hyperbolic vector space 
and has a connected Gram graph. 

Proof. If 7W is a triangle, at least one angle of Ai is not right. It follows that TVf 
has a sequence of consecutive vertices of type (I). If 7W is a quadrangle, we consider 
its narrow place of type (I) from Theorem 4.1.4. If both angles cti = 0^2 = ^, 
we replace 62 by 64 with 5| = —2 which is orthogonal to the line (A0A3) and 
directed outwards of TW. Since a quadrangle has at least one non-right angle (we 
can assume that it is cto = A3AQA1), then the sequence of vertices A2, A3, Aq, 
Ai with orthogonal vectors ^3, ^4, 61 to sides (A2A3), (A3A0), (AqAi) respectively 
satisfies the condition (I) of Theorem 4.1.8. 

We now suppose that AI does not have a sequence of vertices of type (I) (we 
always suppose that it is a sequence of consecutive vertices). Then A^ has more 
than 4 vertices. 

By Theorem 4.1.5, AI has a narrow place of type (II) or (III). 

Suppose that Af has the narrow place of type (II). We have ai = 02 = f since 
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changing the numeration). We then get estimates (4.1.54) and (4.1.55). Obviously, 
the set {5i, ^2, ^3} generates the 3-dimensional hyperbohc vector space and has 
connected components {5i, ^3} and {62} since cti = 02 = f • If the set {5i, ^3, ^4} 
is not connected, it then follows that ^4 is orthogonal to the set {5i, ^3} and 64 = Xd2 
which is impossible since the lines {A1A2) and (A3A4) are different. It follows that 
the Gram graph of the set {^i, ^3, ^4} is connected. This set generates the 3- 
dimensional hyperbolic vector space because the orthogonal lines (AqAi), (A2A3), 
(A3A4) do not have a common point and are not orthogonal to one line. Thus, we 
have proved that the Ai has type (II) of Theorem 4.1.8. 

Assume that A^ has a narrow place of type (III) of Theorem 4.1.5. If the sequence 
Aq, Ai, A2, As, A4, A5 does not have a subsequence of the type (I) of Theorem 
4.1.8, then all angles ai = a2 = a3 = a4 = f are right. We then get estimates 
(4.1.56), (4.1.57) and (4.1.58). The same consideration as above, shows that the 
Gram graph of the {5i, ^3, ^5} is connected and this set generates the 3-dimensional 
hyperbolic vector space. Thus, the Ai has type (III) of Theorem 4.1.8. 

It finishes the proof. 

4.2. Narrow places of restricted parabolic convex polygons on the hy- 
perbolic plane. 

Compare [N9], [Nil]. 

We use notation of Sect. 1.1. Fix a point O = R^+c at infinity of C = C{^). 
Thus, ce^, (c, c) = and (c, V+) > 0. 

Recall that a horosphere Sq with the center O is the set of all lines in C containing 
O. The line / = OR++h e Sq: R++h G £ is the set / = {M++(tc + h) \ t e 
M and {tc + h, tc + h) > 0}. Fix a constant R> 0. Then there is a unique M+^/i G / 
such that (/i, c) = R and (/i, h) = 1. Given /i, I2 G So, we denote the corresponding 
/I's by hi, /i2 and put 

p{lij2) = V-ihi-h2)^. (4.2.1) 

Endowed with this distance, the horosphere So becomes an affine Euclidean space. 
If one changes R, the distance p is multiplied by a constant. We denote 

g^j:, = {R^^h eC\{h,c)=R and (/i, h) = 1}. (4.2.2) 

The set So,r U {O} is a sphere in £, which is tangent to £00 at O. Moreover, 

the set So,R is orthogonal to every line / G £^0 at the point ]R_|_+/i, h G So,r that 
corresponds to /. The distance in C induces Euclidean distance in So,r which is 

homothetic to the distance (4.2.1). The set So,r is identified with So and is also 
called a horosphere. 
Let K cSo- The set 

Ck= [j I (4.2.3) 

line leK 

is called the cone with vertex O and base K. 

A non-degenerate convex locally finite polyhedron A1 in £ is called parabolic 
(relative to the point O G £00 if 1) and 2) below are valid: 

1) A4 is finite at the point O, that is, the set {d G P{A4) \ (c, d) = 0} is finite; 

2) for every elliptic polyhedron M d So (that is M is the convex hull of a finite 
subset of So), the polyhedron M. fi CV is elliptic. 

A „. — „u„l;„ U.I 1 \A ;„ „„11 — J ^„„j.^A„j.„J .„!,„?„■„ ;f 
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3) the set 



r{M) = {(c, 6/V-S^) I S e P{M)} is finite. (4.2.4) 

Geometrically this means that all hyperplanes Tis, S G P{Ai), of faces of Ai are 
tangent to a finite set of horospheres with the center O. 

We remark that li O ^ M. for a parabolic polyhedron M., then M. is elliptic. 
Thus, it is only interesting to consider parabolic polyhedra relative to a point 
O G Coo n M.- Moreover, if M. is parabolic and has a finite number of faces (or it 
has a finite set P{M.)) then M. is elliptic. Thus, it is only interesting to consider 
infinite (i.e. having infinite number of faces) parabolic polyhedra. 

Theorems AAA, 4.1.5 and 4.1.8 can be generalized on restricted parabolic poly- 
gons with strong inequalities replaced by non-strong ones. 

Theorem 4.2.1 (about the narrow place of type (I)). For any restricted 
parabolic convex polygon M. on a hyperbolic plane there exist its four consecutive 
vertices Aq, Ai, A2 and A3 (where Aq — As if M is a triangle) such that for 
orthogonal vectors di, 82 and 5^ to lines [AqAi], {A1A2) and (A2A3) respectively 
directed outwards of Ai and with Sf = 62 = S^ — —2 one has {Si, 52) — 2cosq;i, 
('^2,'^3) = 2cosq;2 and 

either (61,63) < 2 or {61,63) < 4{cos^ + cos^f - 2 < U (4.2.5) 

where cti = A0A1A2 and 0^2 = A1A2A3. 

Moreover, the Gram graph of {61, 62, 63} is not connected (i.e. this set is union 
of two non-empty orthogonal subsets) if and only z/cti = a2 = f • 

Theorem 4.2.2 (about narrow places of types (II) and (III)). For any re- 
stricted parabolic convex polygon M. having more than 3 vertices (i.e. it is different 
from a triangle) on a hyperbolic plane, one of two possibilities (11) or (III) below is 
valid: 

(11) There exist its five consecutive vertices Aq, Ai, A2 , A3 and A4 (where 
Aq = A4 if Ai is a quadrangle) such that for orthogonal vectors 61, 62, 63 and 64 
to lines (AqAi), {A1A2), {A2A3) and {A3A4) respectively directed outwards of Ai 
and with 6^ = 62 = 63 = 6l = —2, one has {6i,6i^i) = 2cosai, i = 1,2,3, 

either {61,63) < 2 or {61,63) < 4{cos^ + cos^)^ - 2 < U, (4.2.6) 

{61,64) < 

ai + (l-ai)t2 + y/a2 + (1 - 02)^2 + ^03 + ast + t'^/4 

4 max -^^ 

o<t<i l + t 

4 max 



t" 



(f ai «2 a3^^ (2 + y/2cos2^ + i)'-i\ 

cos h COS h COS — , ^ 

V 2 2 2 / ' 2 

V / 



-2 <34 

(4.2.7) 



where ai = Ai^iAiAi^i, i = I, 2, 3, and ai = cos^ ^. Moreover, the set {61, 62, 
x j: 1 1 — . „ „ .J. — 7 /^„„,™ .„„!, 
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(HI). There exist its six consecutive vertices Aq, Ai, A2, A3, A4 and A^ (where 
Aq = A5 ifM. is a pentagon) such that for orthogonal vectors di, 62, ^3, ^4 and 6^ to 
lines [AqAi], {A1A2), (A2A3), (A3A4) and (A4A5) respectively directed outwards 
of M. and with 5^ = 5| = ^f = 5| = ^5 = —2 one has ((5^, (5i_(_i) = 2cosQ;i, 
i = l,2,3,4, 

either {5i,5:i)<2 or (Ji, ^3) < 4(cos ^ + cos ^)2 - 2 < 14, (4.2.8) 

either [5:^,5^) < 2 or (^3, ^5) < 4(cos — + cos — )2 - 2 < 14, (4.2.9) 



and 



\ther (53,55)<2 or (^3, ^5) < 4(cos — + cos — )2 - 2 < 14, 

(<Jl,^5)< 



4 max ( ( Jai + (1 - ai)s'^ + A/a2 + (1 - 02)5^+ 
o<t<s<i LVV V ^ 

/ / N U'iis — tY {1 — a^){s + tY I — — „ 

+ \/a3 + a3(s - t) + ^ ^ ' + ^ ^4^ '- + Va4 + (1 - 04)^2 



{s-t)' 



/((l + .)(l + t)) 



-2 



4 max 



OL\ 0,2 Ct3 Ct4 

COS h COS h COS h COS — 

2 2 2 2 



(2 + ^2 cos2 ^ + i + cos 



-2 < 62 



(4.2.10) 



where ai = Ai^iAiAi^i, z = 1, 2, 3, 4, and ai = cos^ ^. Moreover, the set 
{Si J ^2, ^3, ^4, ^5} has a connected Gram graph. 

Theorem 4.2.3. Let M. he a restricted parabolic convex polygon on a hyperbolic 
plane. Then A4 has one of types (I), (11) or (III) (or the type (I), (II) or (III) of 
its narrow place) below: 

Type (I): There exist its four consecutive vertices Aq, Ai, A2, A3 (where Aq = 
A3 if M. is a triangle) with angles cti = A0A1A2, 0,2 = A1A2A3 7^ ^ and such that 
for orthogonal vectors di, 62 and 6^ to lines [AqAi], (A1A2) anc? (A2A3) respectively 

-2 one has (Ji, ^2) = 2cosai, 



H 



directed outwards of M. and with df = 62 
(^2, ^3) = 2 cos q;2 7^ and 

either {Si, 63) < 2 or {Si, S3) < 4{cos^ + cos^f - 2 < U (4.2.11) 

The set {Si, S2, S3} generates the 3- dimensional hyperbolic vector space and has a 
connected Gram graph. Any triangle or quadrangle M. has the type (I). 

Type (II): There exist its five consecutive vertices Aq, Ai, A2, A3, A4^ (where 
Aq = A4^ if M. is a quadrangle) with angles ai = A0A1A2 = f, 0^2 = A1A2A3 = ^, 
as — A2A3A4 and orthogonal vectors Si, S2, S3 and ^4 to lines {AqAi), (A1A2), 
(A2A3) and (A3A4) respectively directed outwards of M. and with Si = S2 — Si — 
5| — —2 such that {Si, S2) = {S2, S3) = 0, (^3, ^4) = 2cosa3, 



t s: s: \ 



I A o 1 o^ 
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and 



4 max 



v 



(v^+COSy) 



(5l, 54) < 



2 

- T \ 

2 <10 + 8v^: 



21.313708... . 
(4.2.13) 



Moreover, the set {5i,53,54} generates the 3- dimensional hyperbolic vector space 
and has a connected Gram graph. 

Type (III). There exist its six consecutive vertices Aq, Ai, A2, A3, A4, A^ 
(where Aq = A^ if A4 is a pentagon) with right angles ai = AqAiA2 = f , a2 = 
A1A2A2, = f , as = A2A3A4 = ^, a4 = A3A4A5 = ^ such that for orthogonal 
vectors di, 62, ^2,, ^4. cind 65 to lines (AqAi), {A1A2), (A2A3), (A3A4) and (^47^5) 
respectively directed outwards of M. and with 5f = 5| = ^3 = 5| = (Jf = —2 one has 
(5i, ^2) = (^2, 53) = ((^3, 54) = (^4, S5) = and 

{Si,Ss)<Q, (4.2.14) 

(^3, 55) < 6, (4.2.15) 

and 

(Ji, 55) < 30. (4.2.16) 

Moreover, the set {Si, Ss, S^} generates the ?>- dimensional hyperbolic vector space 
and has a connected Gram graph. 

Proofs of Theorems 4-2.1, 4.2.2, 4-2.3. If M is parabolic relative to O ^ M, then 
Ai is finite and we can use Theorems 4.1.4, 4.1.5 and 4.1.8. Thus, we can suppose 
that O e M and M is infinite. 

To prove Theorems 4.1.4, 4.1.5 and 4.1.8, we were taken a point O inside of Ai 
and used the formulae of Lemma 4.1.2. To prove Theorems 4.2.1, 4.2.2, 4.2.3, we 
use the point O at infinity of AI such that AI is restricted parabolic relative to O. 
We use an analog of Lemma 4.1.2 which uses the infinite point O. 

Let O — M-|-(-C where c^ = 0. For a line (AB) with terminals A and B at infinity 
and 5 e $ orthogonal to (AB) with 5^ = —2 we introduce an 'angle' 

OiS) = ^ (4.2.17) 

(c, 6) 



where one can replace 6 by —6 according to the orientation of the angle AOB: e. 

it 



one should take 5 such that Tit contains O if AOB is oriented correctly, and 



one should take —5 if not. 

The 'angle' 6{5) really behaves like an angle. For three points A, B and C 
at infinity and three lines {AB), (BC) and (AC) with the corresponding vectors 
5i, 62 and ^3 orthogonal to lines (AB), (BC) and (AC) respectively and with 
51 = $2 = S'^ = —2, one has 

9idi)+e{d2)=9{ds). (4.2.18) 

We leave an elementary proof of (4.2.18) to a reader. 

\'KT„ U„,.„ J-U„ f„11 ; , „„„!„„ „f T „»„»„„ /I 1 o 
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Lemma 4.2.4. Let [AB] and {CD) are two lines on a hyperbolic plane with termi- 
nals A, B , C , D at infinity, and O = M.++C an infinite point on the hyperbolic plane 
which does not belong to each line (AB) and (CD) and orientations of the triangles 
AOB and COD coincide. Let di and 62 are orthogonal vectors with square —2 to 
lines (AB) and (CD) respectively such that O is contained in both half-planes T-fg 
and Til . Let 5\2 be the orthogonal vector with square —2 to the line (BC). Then 

As a corollary, we get : 

1) If lines {AB) and {CD) do not intersect each other, then 

2coshp=.(5i, 52)^4 ^^^^^^ 2 

where p is the distance between lines {AB) and {CD) (here and in what follows we 
normalize the curvature k = —1). 

2) If lines {AB) and {CD) define an angle a containing O , then 

9 (K s^ MSi)-0{S2i)){e{62)-e{62i)) ^ 
2 COS a = [01,02) = 4: 



e{Si)e{62 



where 6{52i) = -9{Si2) = 9{-Si2) 



Proof of Lemma 4-2-4- O^i^ can prove it similarly to Lemma 4.1.2. One can also 
prove it as follows. Take a finite point O' and move O' to the infinite point O. 
Lemma 4.2.4 is the limit of the Lemma 4.1.2 applied to O' when O' tends to O. 
We leave details to a reader. 

Now the proof of Theorems 4.2.1, 4.2.2 and 4.2.3 is the same as proof of the 
corresponding theorems 4.1.4, 4.1.5 and 4.1.8 if one uses the 'angles' (4.2.17) and 
Lemma 4.2.4. One has more: almost all inequalities become equalities, and the 
proof is even simpler. 

4.3. Description of narrow places of fundamental polygons Ai of reflec- 
tion subgroups W C W{S) of elliptic and parabolic type vi^here rk 5" == 3. 
Application to reflective lattices of elliptic and parabolic type. 

Let 5" be a primitive hyperbolic lattice of rk 5" = 3 and W C W{S) its reflection 
subgroup of elliptic or parabolic type with a fundamental polygon JH (see Sects. 
1.3, 1.4). Remind that it means that W and P(A^)pr have restricted arithmetic 
type and P{M.)pT- has a generalized lattice Weyl vector p with p^ > 0. It is known 
(e. g. see [Nil], [N9]) that the polygon A4 is elliptic if p^ > 0, and it is parabolic 
relative to ffi++p if p"^ = 0. We remind that the lattice S having reflection subgroups 
W C W{S) of elliptic or parabolic type is called reflective of elliptic or parabolic 
type. Here we apply Theorem 4.2.3 to describe narrow places of Ai. Using this 
description, we shall get a flnite list of lattices such that any elliptically or parabol- 
ically reflective lattice S belongs to the list. Let a{S* /S) be the exponent of the 
discriminant group S*/S: i. e. a{S) is the least natural a such that aS* /S = 0. 
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Let K he a, lattice. We denote by Kq the primitive lattice defined by K. Thus, 
K = Ko{X) where A G N and Kq is primitive. If K is generated by elements with 
the Gram matrix A, then Kq is generated by the same elements with the Gram 
matrix A/X where A is the greatest common divisor of all elements of A. We denote 
by a{A) the exponent of a finite Abelian group A: i. e. a{A) is the least natural a 
such that aA = 0. 

We have the following useful statement (compare the proof of Theorem 1 in [N5, 
Appendix] ) . 

Proposition 4.3.1. Let L be a primitive lattice and cti, . . . , a^ are primitive roots 
of L which generate a suhlattice G C L of a finite index. Let Gq he the primitive 
lattice defined by G. Then a(L*/L)|8a(G'o/G'o)^- 

Proof. We have G = G'o(A) C L for some natural A G N. Since G is generated 
by roots at of L and a1\2{ai,L), we get 2Go(A) C 2L C AGo(A)*. Identifying 
(naturally) modules of the lattices Go and Go(A), we obviously have AGo(A)* = Gq. 
It follows that 

L = M{\) where Go C M C ^Gq. (4.3.1) 

Here M is any intermediate module which is invariant with respect to reflections 
in roots ai defining the lattice Go, and the roots ai should be primitive in M. The 
A G N is the smallest natural number such that M(A) is a lattice (otherwise, the 
lattice L is not primitive). Using (4.3.1), we get 

2Go C M* c Gl. (4.3.2) 

If tM C M*, t G N, then M{t) is a lattice. Really, for any mi, m2 G M we have 
t(mi,rM2) = (tmi,TO2) G Z because trrii G M*. Using (4.3.1) and (4.3.2), we get 
AaM C M* where a = a{GQ/Go). It follows that A|4a. Identifying modules of M 
and M(A), we have M(A)* = (1/A)M* C (l/4a)G^. It follows that the exponent 
of M(A)*/M(A) divides the exponent of (l/4a)G^/2Go which is equal to Sa^. This 
finishes the proof. 

Below we describe narrow places of the fundamental polygons Ai. According to 
Theorem 4.2.3, a narrow place of Ai is defined by vectors di, ... ,6k with 6f = —2 
where k < 5. We denote by r^ corresponding primitive roots of S such that 

Sr = -^^. (4.3.3) 



-2rt 



with the Gram matrix 



r = (7^,) = ((5.,5,)). (4.3.4) 

Since ri are primitive roots of S, we have rf\2{S, ri). It then follows that 



\2 



«., = il = iS^. S,f = 1%!^ e Z+ (4.3.5) 



r^rj 



are no n- negative integers. 

We want to describe all possible matrices 



D _ f/^. ^.W (A Q a\ 
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where for an integral matrix T we denote by Tpj. the corresponding primitive integral 
matrix T/t where t = g.c.d(T) denote the greatest common divisor of all elements 
of T. We shall make it in three steps. 

First we describe all possible symmetric [k x /c)-matrices 

2 



A = («,,) = ((5„ 5,)') = %^ . (4.3.7) 



The matrix A has non-negative integral coefficients, all its diagonal coefficients are 
equal to 4. All cyclic products 

are perfect squares. 

In the second place, we describe all possible k x k symmetrizable generalized 
Cartan matrices (see [Kac] about generalized Cartan matrices, but remember that 
we use the opposite sign) 

A = (a,) = (%p^) (4.3.9) 

using relations 

an = -2, ttij G Z+ if z 7^ j, aijttji = aij, aij = iff aji = 0, (4.3.10) 

and 

In the third place, we find a diagonal matrix 

A = diag(Ai,...,Afc) (4.3.12) 

with Ai G N such that the matrix 

B' = AA (4.3.13) 

is symmetric. This defines the matrix A uniquely up to multiplication by a scalar 
rational matrix (since the Gram graph of ri, . . . , r^ is connected and the matrix A 
is indecomposable). The matrix A = sdiag(— rf, . . . , — r^) where s G Q++. Then 
we calculate 

which gives the matrix ((r^, r^)) ^, see (4.3.6). These procedure gives a finite set of 
possible matrices B. See the corresponding general considerations in [N5, Appen- 
dix]) 

For the lattice Gq = [ri, . . . , rk]pT defined by B we calculate the invariant a{B) = 
a{GQ/Go), the invariant ai{B) which is the product of all different odd prime 
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of a{B). By Proposition 4.3.1, we get an estimate of the similar invariants a{S), 
ai{S) and a2{S): 



a{S) < 8a{Bf, ai{S) < ai(S), a2{S) < a2{B) 



(4.3.15) 



Calculating the invariants a{B), ai{B) and a2{B) for all possible matrices B and 
taking their maximum, we estimate the invariants a{S), ai{S) and a2{S) for all 
elliptically or parabolically reflective lattices S. 

Below we describe this procedure for each type of the narrow place of Theorem 
4.2.3. 

4.3.1. Matrices B of the narrow places of the type (II). It is a particular case of 
the type (I) of Theorem 4.2.3 when additionally the angle ai 7^ n/2. For this case 
k = 3 and 5i, ^2, ^3 give a bases of the 3-dimensional hyperbolic vector space. We 
get that the matrix ^ is a symmetric matrix 



4 ai2 "13 
A = I a2i 4 ^23 

"31 "32 4 



(4.3.1.1) 



with integral non-negative coeflicients where from the condition of narrow places 
we have (after changing numeration if necessary) that 



1 < «12 = a21 < 4, ai2 < ^23 = «32 < 4, 
«23 < «13 = "31 < 



2 + v^«i2 + \ 2 + y/a^ 



(here we also use that 2 cos ^ = y/2 + 2 cos a) , and 

d = det{T) = -8 + 2^/aI^a23«3i + 2ai2 + 2q;23 + 2ai3 > 

where 

tti2«23a3i is a perfect square. 



(4.3.1.2) 



(4.3.1.3) 



(4.3.1.4) 



It is easy to enumerate the flnite set of all matrices A satisfying (4.3.1.2), (4.3.1.3) 
and (4.3.1.4). For each A we then flnd all symmetrizable generalized Cartan ma- 
trices 

(-2 ai2 ai3\ 
a2i -2 a23 (4.3.1.5) 

«31 ^32 -2 / 

using the relations 

012021 = "12, 023^32 = a23, 013^31 = ai3, 012023^31 = 013032021- (4.3.1.6) 

The diagonal matrix 

A = diag(ai3a32, 023O31, 031032). (4.3.1.7) 

Finally, we get that 



D f A \\ 



I A o t o^ 
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In Appendix, we give the Program 4: fundi 1. gen which uses this algorithm to 
enumerate aU the matrices B. For each of them it calculates the invariants a{B), 
ai{B), a2{B) and finds the number nil of all the matrices B, and the numbers 



all — maxa(i?), alli — maxai(i?), a/l2 = maxa2(-B) 

Calculation using this program gives 

nil = 272, all = 3528, a/li = 543, alh = 181. 



(4.3.1.9) 



(4.3.1.10) 



4-3.2. Matrices B of the narrow places of the type (10). It is a particular case 
of Type (I) of Theorem 4.2.3 when additionally the angle cti = 7r/2. For this case 
k = 3 and 5i, ^2, ^3 give a bases of the 3-dimensional hyperbolic vector space. We 
get that the matrix ^ is a symmetric matrix 



A 



4 





"13 





4 


"23 


"31 


"32 


4 



(4.3.2.1) 



with integral non-negative coefficients where from the condition of narrow places 
we have (after changing numeration if necessary) that 



1 < "23 = "32 < 4, Q;23 < "13 = "31 < 



rv^+y/2 + v/^j -2 



and 



d = det{T) = -8 + 2a23 + 2ai3 > 0. 
For each A we then find all symmetrizable generalized Cartan matrices 

-2 ai3 
A= \ -2 a23 

O3I 0^32 ^2 



using the relations 
The diagonal matrix 

Finally, we get that 



^23^32 — "237 ^13^31 — "13- 



A = diag(ai3a32, 023031, 031032) 



B = (AA)pr. 



(4.3.2.2) 



(4.3.2.3) 



(4.3.2.4) 



(4.3.2.5) 



(4.3.2.6) 



(4.3.2.7) 



In Appendix, we give the Program 5: fundlO.gen which uses this algorithm to 
enumerate all the matrices B. For each of them it calculates the invariants a{B), 
ai{B), 02(5) and finds the number n/0 of all the matrices B, and the numbers 



a/0 = niaxafS), o/Oi = maxaifS), 0/O2 = maxa2(-B). 

B B B 

Calculation using this program gives 

n/0 = 2998, o/O = 69192, a/Oi = 10209, 0/O2 = 89. 



(4.3.2.^ 



(4.3.2.9) 



40 



V.V. NIKULIN 



4.3.3. Matrices B of the narrow places of the type (III). It is a particular case 

of Type (II) of Theorem 4.2.3 when additionally the angle as 7^ 7r/2. For this case 

k = A and 5i, 82, ^3, ^4 generate the 3-dimensional hyperbolic vector space and any 

three of them give a bases of the space. We get that the matrix ^ is a symmetric 

matrix 

/ 4 q;i3 ttMNy 

4 a24 

ttsi 4 Q;34 

\a4i a^2 tt43 4 / 



A 



(4.3.3.1) 



with integral non-negative coefficients where 

1 < "34 = "43 < 4, 4 < Q;i3 = 031 < 36, 

/ 

V2 + 



< ai4 = 041 < 



4 max 



v/^34 1 



2 2 + 



/OiZA I 5 



+ 



2 ' 4 / 4 



l\ 



4 2 



v 



L/2+^j2 + ^/^A -2 



< a24 = «42- 



(4.3.3.2) 



Here we use the conditions (II) of Theorem 4.2.3 and add some inequalities to avoid 
repeating of cases we have considered in Sect. 4.3.3. We also have 



detiV) = 4(4 — v^oisOsiail — Q;i4 — 0^34 — 013) + a24Cii3 ~ 4^24 = 0. (4.3.3.3) 
where 



«i3«34«i4 is a perfect square. 
The condition (4.3.3.3) is equivalent 

4(^/aIiaiiaII + ai4 + 0^34 + ctis - 4) 



^24 



"13 -4 



(4.3.3.4) 



(4.3.3.5) 



We can easily enumerate the finite set of all possible matrices A satisfying these 
conditions. For each A we find all symmetrizable generalized Cartan matrices 



A 



/ -2 ai3 ai4\ 

0-2 a24 

a3i -2 a34 

\a4i 042 G.43 —2/ 



(4.3.3.6) 



using relations 



^34^43 — CI34, ^13^31 — CI137 ^24^42 — «24, 013034041 — Vai3tt34«14, 

014041 = ai4, Oi4 = O41 = if Q;i4 = 0. (4.3.3.7) 

The diagonal matrix 



a;^„I , 



I A O O 0\ 
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Finally, we get that 

B = (AA)pr. (4.3.3.9) 

In Appendix, we give the Program 6: fund21.gen which uses this algorithm to 
enumerate all the matrices B. For each of them it calculates the invariants a{B), 
ai{B), a2{B) and finds the number nlll of all the matrices B, and the numbers 

a//l = max a(i?), a//li = maxai(i?), a//l2 = maxa2(-B). (4.3.3.10) 

B B B 

Calculation using this program gives 

nin = 9818, alll = 47432, allU = 10965, allh = 487. (4.3.3.11) 

4.3.4- Matrices B of the narrow places of the type (110). It is a particular case 

of Type (II) of Theorem 4.2.3 when additionally the angle as = 7r/2. For this case 

k = 4 and Si, ^2, ^s, ^4 generate the 3-dimensional hyperbolic vector space and any 

three of them give a bases of the space. We get that the matrix ^ is a symmetric 

matrix 

/ 4 ai3 q;i4\ 

4 a24 

asi 4 

\a4i a42 4 / 



A = 



(4.3.4.1) 



with integral non-negative coefficients where 



4 < «i3 = asi < 36, < ai4 = a^i < (8 + 4^5)^ = 287.108350... , aia < a24- 

(4.3.4.2) 
(8 + 4a/5 = 16.94427190...) Here we use the conditions (II) of Theorem 4.2.3 and 
that A^ is a fundamental polygon having at least 4 sides. We also have 

det{r) = 4(4 — ai4 — ctis) + a24Cii3 — 4q;24 = (4.3.4.3) 

which is equivalent 

(ai3 - 4) (^24 - 4) = 4ai4. (4.3.4.4) 

It is easy to enumerate the finite set of all possible matrices A satisfying these 
conditions. For each A we find all symmetrizable generalized Cartan matrices 



-2 ai3 ai4N^ 

.0 -2 a24 

' asi 0-2 

M41 a42 —2/ 



(4.3.4.5) 



using relations 

CllSttsi = Ctis, 024^42 = tt24, 014041 = Q;i4, (4.3.4.6) 

The diagonal matrix 

A = diag(ai3ai4a42, 013041024, 031014042, 013O42O41). (4.3.4.7) 

Finally, we get that 

r> ^'/lA^ f A O A 0\ 
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In Appendix, we give the Program 7: fund20.gen which uses this algorithm to 
enumerate aU the matrices B. For each of them it calculates the invariants a{B), 
ai{B), a2{B) and finds the number nllO of all the matrices B, and the numbers 

a//0 = max a(i?), a//Oi = maxai(i?), a//02 = maxa2(-B). (4.3.4.9) 

B B B 

Calculation using this program gives 

nllO = 376208, allO = 995316, allOi = 238569, a/JOs = 283. (4.3.4.10) 

4-3.5. Matrices B of the narrow places of the type (HI). For this case k — 5 and 
^ij ^2 J (^3, ^4, ^5 generate the 3-dimensional hyperbolic vector space, and any three 
of them give a bases of the space. The matrix .A is a symmetric matrix 

/ 4 ais ai4 ai5 \ 

4 ^24 "25 

A= asi 4 035 (4.3.5.1) 

«41 «42 4 

Va5i «52 ass 4 / 
with integral non-negative coefficients where 

4 < ai3 = "31 < 36, ttai < 035 = 053 < 36, < ai5 = ctsi < 30^ = 900, 
287.108350 < om = 041, 287.108350 < ^25 = "52, (4.3.5.2) 

Here we use the conditions (III) of Theorem 4.2.3 and that A1 is a fundamental 
polygon having at least 5 sides. The last two inequalities were added to avoid 
repetition with the previous case. The Gram matrix F — F(5i, 62, ^3, ^4, ^5) has 
the rank 3. Coefficients of the matrix A are defined by the coefficients ais, 015, 
0^35 defining the Gram matrix F(5i, ^3, ^5). Let 

d = 4(q;i3 + 035 + ai5 + V«i3«35«i5 - 4), (4.3.5.3) 

one can see that d/2 = det{r{di, ds, ^5)). We have 

ai4 = T, «25 = 7 (4.3.5.4) 

^35 - 4 ai3 - 4 

and 

4(ai3a35 + 4^/aTiaiiaT^ + 4ai5) /, o r r\ 

«24 = -. TT7 TT 4.3.5.5 

(ai3 -4)(a35 -4) 

Here (4.3.5.4) follows from det(F(5i, ^3, ^5, S2)) = det(F(5i, ^3, ^5, ^4)) = 0. To 
get (4.3.5.5), one should remark that 62 = 725*^5 and ^4 = judl. It follows that 
724 = 725714(5'""^) 13 where g = F(5i, ^3, ^5). All together, (4.3.5.4) and (4.3.5.5) 
are equivalent to rk (F(5i, 62, S3, ^4, ^5)) = 3, or that 5i, 62, Ss, ^4, ^5 generate a 
3-dimensional hyperbolic form. Moreover, cyclic products 
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It is easy to enumerate the finite set of all possible matrices A satisfying these 
conditions. For each A we find all symmetrizable generalized Cartan matrices 



A 



/-2 





ai3 


ai4 


ai5\ 





-2 





a24 


a25 


asi 





-2 





^35 


a4i 


a42 





-2 





Va5i 


a52 


053 





-2/ 



(4.3.5.7) 



using relations 



fll3«31 — Cll35 ^35^53 — C1357 ai4«41 — CI4I7 0^25^*52 — CI257 0^15«51 — CH57 

ai5 = asi = if ais = 0, 024^42 = "24, ^25^52 = "25, a35a53 = «35, 



«13tl350^52fl24fl41 — V'^13Ct35Ct52Ct24Cl41 , ^13^35^51 — vOl3«35Q^7 

(4.3.5.8) 

The diagonal matrix 

A — diag(ai4ai3a42a25, «4ltll3tl24«25 5 «14«3l0^42tl25 5 «4ltll3tl42«255 tl4lfll3tt24fl52) 

(4.3.5.9) 
Finally, we get that 

B = (AA)pr. (4.3.5.10) 

In Appendix, we give the Program 8: fund30.gen which uses this algorithm to 
enumerate all the matrices B. For each of them it calculates the invariants a{B), 
ai{B), 02(5) and finds the number nil I of all the matrices B, and the numbers 

alll = maxa(i?), allli = maxai(i?), alll2 = maxa2(-B). (4.3.5.11) 

B B B 

Calculation using this program gives 

nlll = 200539, alll = 324900, allh = 26565, allh = 907. (4.3.5.12) 

4.3.6. The global estimate of invariants of primitive reflective hyperbolic lattices 
of the rank 3 having elliptic or parabolic type. 

For a lattice L we denote by a{L* /L) the exponent of the discriminant group 
L* /L, we denote by ai{L*/L) the product of all different odd prime divisors of 
a{L* /L), and we denote by a2{L* /L) the greatest prime divisor of a{L* /L). 

Using Proposition 4.3.1 and calculations (4.3.1.10), (4.3.2.9), (4.3.3.11), 
(4.3.4.10), (4.3.5.12) we get 

Theorem 4.3.6.1. For any primitive reflective hyperbolic lattice S of rk S = 3 
having elliptic or parabolic type we have estimates: 

a{S*/S) < 8(995316)^ ai{S*/S) < 238569, a2{S*/S) < 907. 

Since det(S') < a{S* /S)'^ and number of lattices with the fixed rank and deter- 
minant is finite (e. g. see [C]), Theorem 4.3.6.1 gives a finite list of lattices which 
contains all the refiective lattices S. 

The estimates of Theorem 4.3.6.1 are very preliminary, and we shall significantly 
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5. Classification of reflective hyperbolic lattices of 
the rank 3 and of elliptic or parabolic type: proofs 

Now we are ready to prove classification results of Sect. 2. 

5.1. Proof of Basic Theorem 2.3.2.1. 

We first prove 

Theorem 5.1.1. Any elliptically or parabolically reflective main hyperbolic lattice 
S of rank 3 and with square-free determinant belongs to the list of Table 3 containing 
all main hyperbolic lattices S of the rank 3 with square-free d = det(5') < 100000 
and h = hnr{S) < 1. 

Proof. Let S be an elliptically or parabolically reflective main hyperbolic lattice of 
the rank 3 and with square- free determinant d = det(S'). Let A^ be a fundamental 
polygon of W{S). 

First we will show that h{S) < 1 (it gives the proof of Lemma 2.3.1.2). If S is 
elliptically reflective, then Ai is an elliptic (flnite and of flnite volume) polygon. 
This polygon has < 1 central symmetries. Really, otherwise, a composition of two 
different central symmetries gives an automorphism of inflnite order of Ai which is 
impossible. It follows that h{S) < 1. Assume that S is parabolically reflective and 
p is a generalized lattice Weyl vector for Ai. Let A{A4) C 0~^{S) be the group of 
symmetries of Ai. By deflnition of a generalized lattice Weyl vector p & S, then p 
is preserved by a subgroup A C A{Ai) of flnite index, p^ = and p 7^ 0. For any 
(p € A{Ai), the element (p{p) also has all these properties and is then a generalized 
lattice Weyl vector for At. If (j){p) 7^ p, the group A{Ai) has a subgroup of flnite 
index which is trivial on the hyperbolic sublattice Zp + 'L(f){p) C S. It then follows 
that A{Ai) is flnite and 5" is elliptically reflective. We get a contradiction. Thus 
(/>(p) = p for any ^ G A{Ai). If tt G A{Ai) is a central symmetry, then the flxed 
part S'^ = {x E S \ u{x) = x} of u is negative deflnite and does not have non-zero 
elements p with p^ = 0. This shows that h = h{S) = 0. 

Now we apply to S and 7W results of Sect. 4.3. By Proposition 4.3.1 and 
(4.3.1.10), (4.3.2.9), (4.3.3.11), (4.3.5.12), we have d < 100000 if Al has a narrow 
place of types /I, /O, III or ///. Really, for all this cases the invariant ai < 50000 
and then d < 2ai{S) < 2ai < 100000. Thus, Theorem 5.1.1 is valid for these 
lattices 5". 

Only if 7W has a narrow place of type //O, our estimate (4.3.4.10) of ai is not 
good enough: 

ai{S) < allOi = 238569, a2{S) < 238. (5.1.1) 

Let us consider this case. In Sect. 4.3.4, for all primitive elliptically or parabolically 
reflective hyperbolic lattices 5" or rank 3 we considered the primitive Gram matrices 
B = Fpr where F = F(ai, q;2, cts, «4) is the Gram matrix of primitive roots of S 
corresponding to a narrow place of type 1 10 of A^. We now consider them only for 
main lattices 5" with square-free d = det(5'). Then we should add some additional 
conditions to Sect. 4.3.4. We introduce these conditions below. 
Let (3 E S he a, primitive root and K = S^. Then 

either S = Z/3 ® K or S = [P, K,{/3 + k)/2] where k e K. (5.1.2) 

It follows (by a simple consideration over Z2) that 

a2 ■ „ . e /r 1 o\ 
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If /3i, /32 G S" are two primitive orthogonal roots and Z/i = [/9i, /32]^, then for any 
odd p 

S ^ Zp = Zp/3i © Zp/32 © Zph and /i^ is square-free. (5.1.4) 

It foUows 

g.c.d.{(3l (3i) < 2, g.c.d.{(3l h^) < 2, g.c.d.(/3|, /i^) < 2, (5.1.5) 

and 

d = l3ll3lh^/2\ where t = mod 2. (5.1.6) 



(-1) 



_ 7 



It defines the t uniquely because d is square-free. From (5.1.6), for any odd p\d, we 
also have 

P 
P 

V 



iip\0l 

-^^P\^l (5.1.7) 

if p|/i^ 



where ry is the invariant of S. The Gram matrix F = F(q;i, 0^2 , 03, 04) has three 
pairs of orthogonal roots: (cti, 0^2) = (ct2, cts) = (ci3, ct4) = 0. It then follows (it is 
sufficient to have only one pair ) that 

either F = S or F = 2S. (5.1.8) 

By (5.1.3), in (5.1.8), the second case is possible only if 611 = 622 = ^ss = ^44 = 1 
mod 2. We denote (like in Sect. 4.3) by Gq the lattice defined by the primitive 
matrix B. Suppose that 611 = 622 = ^33 = ^44 = 1 niod 2 and det(Go)) — 2^to 
where m is odd and /c = 1 mod 2. We then have the second case F = 2B in (5.1.8), 
because otherwise 5" is odd but d is even. Thus, for a fixed S, we may have both 
cases in (5.1.8) only if hn = 622 = ^33 = ^44 = 1 rnod 2 and det(Go) = 2'^rM where 
m is odd and k = mod 2. In all other cases the matrix B prescribes the Gram 
matrix F. For both cases in (5.1.8), we can calculate d using the matrix B since 
d is square-free. If one of ctf is odd but d is even, then F = S, and this case is 
impossible for a main S (the lattice S should be even for even d). 

Summarizing considerations above, we get the following additional conditions 
for B and we calculate the invariants {d, i]) of the lattice 5" using the matrix B. We 
denote by Gq the lattice with the matrix B. We have 

^11, ^22, ^33, ^44 are square-free (5.1.9) 

and 

g.c.d(6ii, 622) < 2, g.c.d(622, 633) < 2, g.c.d(633, ^44) < 2. (5.1.10) 

We denote by iy'p{a) the order of p in factorisation of a. We have 

if odd p|6ii622^33^44, then z/p(det(G'o)) = 1 mod 2. (5.1.11) 

We have 

F = S, if not all 611, 622, ^33, ^44 are odd; (5.1.12) 

F = 2i?, if 611 = 622 = ^33 = ^44 = 1 rnod 2 and U2{det{Go)) = 1 mod 2; 

('CI 1 o^ 
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r == S or r = 2B, if 611 = 622 = ^33 = ^44 = 1 mod 2 and U2{det{Go)) = mod 2. 

(5.1.14) 
We denote by G the lattice generated by ai and defined by the Gram matrix 
r = ((cti, cxj)). Let (i be the product of aUp such that p I det(G) and Up{det{G)) = 1 
mod 2. We have 

z/2((i) = mod 2 if one of of is odd. (5.1.15) 

For odd p I (i we have 



/ 



(-1)"^ = < 



"^^"^ ifp\al (5.1.16) 

otherwise. 



p 

2 2 



'^QlQ2/P 
P 



This defines the invariant r]. At last, we have the most delicate condition which 
enormously drops the finite number of possibilities: 

h = hnr{d,r])<l (5.1.17) 

(see Theorem 3.2.1 about hnr{d, rf)). To calculate hnr{d, t]) using Theorem 3.2.1, we 
should calculate class-numbers of imaginary quadratic fields of the discriminant D 
where -4d < D < 0. By (5.1.1), d < 2-238569 and < -D < 8-238569 = 1908552. 
Moreover, by (5.1.1), prime divisors of D are not more than 238. Thus, checking the 
condition (5.1.17), we work with reasonable (not too big) integers. Or, if one wants, 
we should use a program which correctly calculates class-numbers of discriminants 
D where —1908552 < D < and D is product of primes p < 238. 

In Appendix: Program 9: fund20.main, we give a program for GP/PARI calcu- 
lator, which enumerates all the matrices B satisfying the conditions of Sect. 4.3 
(this part of Program 9 is the same as the Program 7: fund20.gen) and conditions 
(5.1.9) — (5.1.17), and gives all triplets of invariants (d^rj^h) of the matrices B 
(it gives 132 triplets). We can see that all these triplets belong to the Table 3. 
The GP/PARI calculator calculates class numbers of negative discriminants D for 
\D\ < 10^^ (see user's guide to the calculator). This finishes the proof of Theorem 
5.1.1. 

By Theorem 5.1.1, to prove Theorem 2.3.2.1, we should now check refiective 
type of all lattices of Table 3 and calculate the sets P{Ai)p^ and the Gram matrices 
G{P{Ai)pT:) if they are elliptically or parabolically refiective. In Table 3, for each 
pair of invariants (d, rf) we give a main lattice 5" with these invariants. If 



S = u®{-d), (5.1.18) 



we have 



(-1)"'' = ( ^^ J , for any odd p\d. (5.1.19) 

If 
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where ei,e2,e3 G {0, 1}, we have d = nin2'n^ if ei = £2 = ^3 = 0, and d = 
nin2n3/4 otherwise. Moreover, n^ = mod 2 if e^ = 1; (niei — 71262 —'n,3e3)/4 G Z; 
(niei — ?i2e2 — n3e3)/4 G 2Z if (i = mod 2; for any odd p\d we have 



(-1)' 



211Z \ if p\n 



1; 



^) ifp|n2, (5.1.21) 

^^^^"^ ifp|n3. 



Checking aU these conditions, one can prove that our calculation of main lattices 5" 
corresponding to the invariants {d, r]) of Table 3 are correct. Using known criteria 
(e. g. see [Se]), it is easy to prove that S = U (B {—d) if and only if 5" represents 0. 
The condition (5.1.19) is equivalent to this property. If (5.1.19) is valid, we always 
give 5" in the form S = U Q) {—d). 

Thus, fortunately, all our lattices S have one of two forms (5.1.18) or (5.1.20). 

We use the Vinberg's algorithm [V2] to calculate the sets P{A4)pr for lattices of 
the forms (5.1.18) and (5.1.20). Below we describe this algorithm. 

First, we remark that the lattice U © {—d) is equivariantly equivalent to its 
maximal even sublattice U © {—^d) if d is odd. These lattices have naturally 
isomorphic groups of automorphisms and the reflection groups. Thus, it is sufficient 
to consider only the lattices 

U®{-2k), ken. (5.1.22) 

For lattices S of the form (5.1.22), we use the isotropic vector c = (1, 0, 0) as the 
center of Vinberg's algorithm. We find M. which contains M+^c as an infinite vertex 
and vi = (0, 0, 1), ^2 = {n, 0, —1) as roots orthogonal to faces of A^ containing ]R_|_+c 
(equivalently, they are roots of the height 0). It is easy to see that t'i,t'2 give the 
orthogonal primitive roots to the fundamental polyhedron of the stabilizer subgroup 
of c in the refiection group W{S). 

For lattices 5" of the form (5.1.20) we assume that either e = (ei, £2, e3)/2 = or 
ej 7^ at least for two different i. We take c = (1, 0, 0) as the center of Vinberg's 
algorithm. We take as roots of the height 
vi = (0, 1, 0) and V2 = (0, 0, 1) if n2/n3 ^ 1/3, 1, 3; 
vi = (0, 1, 0), ^;2 = (0, -1, 1) if n2 = ng and e = (0, 0, 0), (1/2, 1/2, 1/2); 
vi = (0, 1, 0), 1^2 = (0, -1/2, 1/2) if n2 = 713 and e = (0, 1/2, 1/2); 
vi = (0, 1, 0), ^2 = (0, 0, 1) if 712 = ns and e ^ (0, 0, 0), (1/2, 1/2, 1/2), (0, 1/2, 1/2); 
vi = (0, 1, 0), t;2 = (0, -1/2, 1/2) if n2/n3 = 3 and e = (0, 1/2, 1/2); 
vi = (0, 1, 0), t;2 = (0, 0, 1) if n2/n3 = 3 and e ^ (0, 1/2, 1/2); 
vi = (0, 0, 1), 1^2 = (0, 1/2, -1/2) if n2/n3 = 1/3 and e = (0, 1/2, 1/2); 
vi = (0, 1, 0), 1^2 = (0, 0, 1) if n2/n3 = 1/3 and e ^ (0, 1/2, 1/2). 
One can check that t'i,t'2 are the orthogonal primitive roots to the fundamental 
polyhedron of the stabilizer subgroup of c in W{S). 

Further steps of the Vinberg's algorithm are prescribed canonically (see [V2]). 
One introduces the height 

heieht = ^ \J G Z f5.1.23l 
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of primitive roots d E S. The roots {vi, V2} above give all roots of P{M.)pj- of 
the height 0. If one knows all roots of P(A^)pf "^ C P(A^)pi- of the height < n, all 
primitive roots P{Ai)J^~^^ C P{Ai)pr of the height n + 1 are given by the condition 

^^^\^J = n + 1 and (5, P(7W)^7") > 0. (5.1.24) 

—0 

In Appendix, Program 10: reflO.l, we give this algorithm for lattices (5.1.22). In 
Appendix, Program 12: reflO.13, we give this algorithm for lattices (5.1.20). We 
also give Program 11: refl0.12 which calculates only for ei = 62 = €3 = but 
is faster. These programs calculate a sequence Vi of all elements of the subset 
P(A<)p7" C PiM)pr (i. e. aU roots of the height < n). 

After calculation of P{A4)^^ for a sufficiently large height n, next steps of 
our algorithm are as follows. A sequence r = ri, . . . ,rfc of different elements from 
P{Ai)pj- is called a chain if 

< ^^^'^^^'^ ^2, for any 1 < z < A; - 1. (5.1.25) 

Geometrically it means that Ti^i 7 • • • 7 ^rfc are lines of consecutive sides (i. e. defin- 
ing vertices) of the polygon Ai. We find the maximal chain e = (ei,...,efc) in 
P(A^)p7" containing vi. Suppose that (for a sufficiently large n) we also have 

4(ei,efc)2 



p2 2 



< 2, (5.1.26) 



i. e. the lines He^ and Tiei ^^^o define a vertex of Ai. Then the polygon A4 
is elliptic, P{A4)pT- — e, and the chain e gives the orthogonal primitive roots to 
consecutive sides of the elliptic polygon A4. This situation takes place for 122 
invariants {d,r]) of the Table 3 marked by er (elliptically refiective cases). The 
result of our calculation of the chain e and the Gram matrix ((ei,ej)), e^, Cj G e, 
for these 122 cases is given in Table 1. 

For all other 206 — 122 = 84 cases we should prove that 5" is not refiective of 
elliptic or parabolic type. 

To prove that, for small determinants d we use the following arguments. For a 
large height n, the chain e C P^M.)^"^ contains a period: there exists 1 < q < k 
such that the Gram matrices r(ei, 62) and r(eg, e^+i) coincide and there exists 
C e 0+{S) such that C(ei) = e,, C(e2) = e^+i. It follows that C e A{M). 
We find all integral (i.e. from 5") eigenvectors u of C, and show that all of them 
have -u^ < 0. Clearly, these eigenvectors have the eigenvalue ±1. It follows that 
5" is not elliptically or parabolically refiective. Really, if 5" is elliptically refiective, 
the group A{Ai) is finite, C has finite order and has an integral eigenvector with 
the eigenvalue 1 and positive square. If S is parabolically refiective, then A has an 
eigenvector with the eigenvalue 1 and with zero square (the generalized lattice Weyl 
vector p). See the proof above of Lemma 2.3.1.2. If 5" is hyperbolically refiective, 
the eigenvector u defines a generalized lattice Weyl vector p. 

Let us consider an example of these calculations for {d,r]) = (114,2). Then 
S = U Q) (—114). Using Program 10 we calculate up to the height n = 50000 and 
find the chain e in P(7Vf) f " which is equal to 

ei = (321, 30, -13), 62 = (28, 2, -1), eg = (57, 0, -1), 64 = (0, 0, 1), 

„ / 11 ^^ „ la a 1^ „ /'ono 100 oc;^ I r. 1 o'7^ 
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with the Gram matrix 



ge 



( -6 





228 


1482 


291 


714 


10872 \ 





-2 





114 


26 


72 


1150 


228 





-114 


114 


57 


228 


4104 


1482 


114 


114 


-114 





114 


2850 


291 


26 


57 





-2 


3 


170 


714 


72 


228 


114 


3 


-6 





V 10872 


1150 


4104 


2850 


170 





-2 / 



(5.1.28) 



We see that the Gram matrices of ei, 62 and ee, e-j coincide. We calculate that 

2209 22800 107160 \ 
C=\ 912 9409 44232 (5.1.29) 

-188 -1940 -9119/ 

belongs to 0+(5') and C{e\,e\) = (65,67). It follows that C G A{M). The matrix 
C has the only integral eigenvector w = (95, 19, —6), it has the eigenvalue 1. We 
have w'^ = —494. It follows that S is not elliptically or parabolically reflective: the 
automorphism C has infinite order and does not have integral eigenvectors with 
non-negative square. 

Since for ((i, 77) = (114,2) the invariant /i = 0, it is possible that 5" is hyperbol- 
ically reflective with the generalized lattice Weyl vector p = w. Let us prove that 
this is the case. We flnd another chain / in P(A^) f " which is equal to 

/i = (1766,172,-73), /a = (6384,627,-265), fs = (4560,456,-191), 

/4 = (283, 29, -12), /5 = (18, 3,-1), /e = (14,4, -1), 

fr = (456, 171, -37), /g = (2280, 912, -191), /g = (427, 173, -36) 

(5.1.30) 



with the Gram matrix 



/ 



gf 



-2 


114 

26 

72 

1150 

72504 





-114 

114 

57 

228 

4104 

259806 



114 

114 

-114 



114 

2850 

182058 



26 
57 


-2 

3 

170 

11001 



72 1150 72504 413250 79370 \ 
228 4104 259806 1481658 284601 ' 
114 2850 182058 1039566 199728 



3 

-6 


228 



413250 1481658 1039566 62928 1482 
\ 79370 284601 199728 12094 291 



170 


-2 


114 
26 



11001 
228 



-114 

114 

57 



62928 

1482 
114 
114 

-114 




12094 
291 
26 
57 

-2 



(5.1.31) 



/2 is equal to the Gram matrix of /e, A. We 
Moreover, we can see that {ei,w) > and 



We see that the Gram matrix of /i 
calculate that C(/*,/*) = (/e*,/^). 
(/^,H<0. 

It follows that P{A4)pr = H{e) UH{f) where H = [C] is the infinite cyclic group 
generated by the C. The infinite polygon Ai contains the line (the axis) Tiw which 
is preserved by H. All sides of A4 define two infinite chains TYj, d G H{e) and 
Hs, S G H{f), these two chains are contained in two different half-planes bounded 
by the line Tiw The polygon Ai is finite in every orthogonal cylinder over a 
compact base in TCw (it is restricted hyperbolic relative to Hw)- The polygon Ai 



f/^i 



,j. j.U„ A, 11 



A /■ A yl\ 
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of A^. After considerations above, it is easy to see that A{A4) is the cychc group 
generated by the ghde reflection Ci with axis Hw where 

/ 48 475 2280 \ 
Ci = 19 192 912 . (5.1.32) 

y-4 -40 -191/ 

The vector w is the eigenvector of Ci with the eigenvalue —1. We have Cf = C. 
The glide reflection Ci changes places the infinite chains H{e) and H{f). The 
lattice 5" is hyperbolically reflective with the generalized lattice Weyl vector p = w. 

For large d following arguments are very useful. We calculate the set P{A4)^^ 
for a sufficiently large height n and find two pairs of elements ri,r2 G P(7W)pf", 
si,S2 e P{M)py'^ such that 4{ri,r2)'^ /rfrj < 2, 4(si, S2)^/s?S2 < 2 and Gram 
matrices of these pairs coincide. In many cases it is sufficient to consider pairs with 
iri,r2) — (si,S2) — and rf = sf, r| = 83. Thus, orthogonal sides of these pairs 
define two vertices of Ai. We find such pairs that there exists an automorphism 
B G 0~^{S) such that B{ri) = Si. Then B G A{A4). Similarly, considering two 
other pairs in P(A^)pf '^, we find another C G A{M). If S^C^ ^ C^B^, the lattice 
5" is not refiective of any type: elliptic, parabolic or hyperbolic. In many cases it 
is sufficient to find only one B G A{Ai) and calculate that B^^ 7^ E. Really, then 
B has infinite order and the lattice S cannot be elliptically refiective. If h = 1, the 
lattice S also cannot be parabolically or hyperbolically reflective. 

We consider an example of these calculations for {d,rj) = (3990,4). Then 5" = 
(30) © (-38) © (-14) (1/2, 1/2,0). Using Program 12 from Appendix, we calculate 
up to the height n = 500000. It gives 33 elements Vi G P(A^)pf ". They are 

VI = (0, 1, 0), z;2 = (0, 0, 1), t;3 = (1/2, -1/2, 0), 

^4 = (2, 0, -3), V5 = (13/2, -9/2, -6), ve = (7, -3, -9), 

V7 = (57/2, -21/2, -38), vs = (17/2, -9/2, -10), v^ = (28, -21, -22), 

^10 = (63/2, -35/2, -36), i^n = (57, -37, -57), Vi2 = (25, -15, -27), 

^13 = (42, -14, -57), z;i4 = (17, -4, -24), vi^ = (76, -58, -57), 

Vie = (19, -6, -26), vu = (52, -15, -72), vis = (84, -21, -118), 

t'lg = (1729/2, -1015/2, -950), t;20 = (119/2, -69/2, -66), 

^21 = (69/2, -43/2, -36), 

^22 = (73, -57, -51), z;23 = (74, -57, -54), V24 = (231/2, -147/2, -118), 

^;25 = (2261/2, -1645/2, -950), t;26 = (101, -27, -141), i;27 = (266, -132, -323), 

1^28 = (119, -87, -99), z;29 = (128, -63, -156), z;30 = (399/2, -315/2, -134), 

t;3i = (342, -96, -475), 1^32 = (361, -77, -513), v^s = (238, -119, -288). 

(5.1.33) 

We look at all the pairs Vi, Vj with {vi, Vj) = 0, and we flnd that there are flve such 
pairs t'i,i'2; f7,f^i3; vi5,V9; i'ii,t'24; i'27,^'33 having squares vf = -38, v'j = -14. 
We calculate a primitive orthogonal element w E S to each of these flve pairs Vi, Vj, 
and we flnd that between these flve pairs there are two pairs fii,f24 and f 27, 1^33 
such that {w + Vj)/2 G 5". We then flnd the matrix 

/ 6863/2 5339/2 1694 \ 
C= -3345/2 -2601/2 -826 (5.1.34) 
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such that C G 0+(5') and C{v\^,v\^) = (^27,^^33)- It follows that C G A{M). We 
have C^^ 7^ E. It follows that C has infinite order. Thus, the lattice 5" is not 
elliptically refiective. For ((i, ?]) = (3990,4), the invariant /i = 1, thus the lattice 
5" cannot be also parabolically or hyperbolically refiective. It follows that S is not 
refiective. 

These arguments permit to prove that the rest 84 cases of Table 3 which are not 
contained in the Table 1, are not refiective of elliptic or parabolic type. It finishes 
the proof. 

Remark 5.1.2. Here we want to outline another way which helps to study refiec- 
tive type of lattices of Table 3. We can write similar programs as Program 9: 
fund20.main for all types of narrow places of M. of main lattices S having the in- 
variant h < 1. They would be programs fundll.main, fundlO.main, fund21.main 
and fund 30. main specializing the programs fundll.gen, fundlO.gen, fund21.gen and 
fund30.gen (one can write them similarly to Program 9: fund20.main). Together 
with Program 9: fund20.main, they give a list L of invariants {d, ry, h) containing 
in Table 3. All invariants {d, rj, h) of Table 3 which are not in the list L cannot 
be elliptically or parabolically refiective. If /i — 1, they cannot be hyperbolically 
refiective either. 

We did this calculations and we found that the list L is much smaller than Table 
3 (e. g. Program 9: fund20.main gives only 132 triplets {d,r],h)). It shows that 
the narrow places of polyhedra arguments are sometimes stronger than arithmetic 
arguments of studying the invariant h (even if we forget about the problem with 
infinity). Both these arguments surprisingly fit together. 

Here is the list of invariants {d, rj, h) which are contained in Table 3 but are not 
in the list L (39 pairs): 



{d,r],h) = 
(57,1,1); 
(194,0,1) 
(299,3,1) 
(506,0,1) 
(714,6,1) 
(966,2,1) 
(3311,1,1; 



(65,3,1); (71,0,1); (119,3,1 



(246,0,1): 
(326,0,1): 

(530,0,1): 

(777,6,1): 
(1106,1,1 
; (3990,4, 



(259,3,1 
(335,0,1 
(534,0,1 
(854, 3 
(1254 
); (4466 



1 

4,1) 
1,1 



(161,1,1 
(266,3,1) 
(386,0,1) 
(602,3,1) 
(897,7,1) 
(1394,3,1); (1659 



(182,3,1) 



(266,0,1 
(354,2,1 
(546,2,0 

(897,5,1 



(285,1,1) 
(407,0,1) 
(645,6,1) 
(935,6,1) 
2,1); (2210,6,1); 

(5.1.35) 



These {d, i], h) give lattices S which are not elliptically or parabolically refiective: 
their fundamental polygon Ai does not have a narrow place satisfying Theorem 
4.2.3. Of course, the list (5.1.35) is in complete agreement with calculations above 
using Vinberg's algorithm. 

5.2. Proof of Theorem 2.3.3.1. 

We should check refiective type of non-main lattices S corresponding to main 
lattices S of Table 1 with odd d (there are 97 these cases). See Proposition 2.2.6. 
Let S be one of lattices of Table 1 (or of Table 3 marked by er) with invariants 
((i, 77) where d is odd. Then S has invariants {2d, odd,!] + uj{p)). If 5" and 5" are 
equivariantly equivalent, i. e. 



y {1 — p + Arjp + 4a;(p)) = or 6 mod 8, 



(5.2.1) 
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then lattices 5" and S have the same reflective type and calculation of P{A4)pj- and 
its Gram matrix for the lattice S follows from similar calculation for S (see Remark 
2.3.3.2). There are 21 these cases. For example if S = U (B {—d), where d is odd, 
then S = (1) Q) (—1) © {—2d), and these lattices are equivariantly equivalent. Thus, 
we need to study only cases when (5.2.1) is not valid. There are 97 — 21 = 76 these 
cases. Let S be one of these lattices. 

We have: if 5" = (?ii) © (— ^^2) © (— "^-3), where all rii are odd, then 5" = (2?ii) © 
(— 2?i2) © (— 2?i3)(ei, £2, €3) where one of e^ is equal to 0, two of them are 1/2 and 
(eini - e2n2 - 63713)72 is odd. If 5" = (ni) © (-^2) © (-?i3)(ei, £2, £3) where for 
example ei = 62 = 1/2 and £3 = (it follows that ni = 712 = 2 mod 4 and 
n3 is odd), then S = (ni/2) © (-n2/2) © {-2ns). We see that the lattice S 
which we should check for reflective type, has the form (5.1.20). Thus, we need to 
make similar calculations as in Sect. 5.1. They are of the same difficulty. These 
calculations finish the proof. 

6. Appendix: Programs for GP/PARI calculator 

Program 1: h2 

\\hclass(d,muu) calculates h=(hrl(d,muu),hrll(d,muu),hnr(d,muu)) 

\\here d<0 and 0\le muu<2^k are integers, 

\\k is the number of all odd prime divisors of d. 

\\Assume d is a fundamental discriminant of fundamental binary 

Wpositive lattices (i.e. with a square free determinant), 

\\then d\equiv 1 \mod 4 or 

\\d \equiv \pm 4, 8\mod 16. 

\\ Assume (d,muu) is the genus of them. Then 

\\hrl(d,muu), hrll(d,muu), hnr(d,muu) are numbers of ambiguous 

\\classes of the types I, II and 

\\hnr (d,muu) =(h(d) /2^ {tau(d) }- hrl(d,muu)-hrll(d,muu) ) /2 

\\of non-ambiguous classes of the general 

\\equivalence respectively of the genus (d,muu); 

\\hclass(d,muu) = [0,0,0] if 

\\d is not a fundamental discriminant. 

\\if d is a fundamental discriminant but 0\le muu <2^k 

\\does not correspond to a 

Wgenus, then hnr(d,muu) = [0,0,h(d)/2^{tau(d)+l}] 

hclass(d,muu,dd,fdd,ddl,beta,alpha,k,hr,hrl,hrll,t,h)=\ 

h= [0,0,0] ;hr=0;hrl=0;hrll=0;\ 

if(mod(d,4)!=mod(l,4)„dd=-d;\ 

fdd=factor(dd);\ 

fordiv(dd,ddl,if(ddl>dd/ddl„\ 

beta=l;alpha=l;k=l;\ 

while ( alpha, \ 

if(type(ddl/fdd[k,l])==l,\ 

if(kro(2*ddl/fdd[k,l],fdd[k,l])==(-l)^bittest(muu,k-l), \ 

if(k>=matsize(fdd)[l],alpha=0,k=k+l),beta=0;alpha=0),\ 

if(kro(2*(dd/ddl)/fdd[k,l],fdd[k,l])==(-l)^bittest(muu,k-l),\ 

if(k>=matsize(fdd)[l],alpha=0,k=k+l),beta=0;alpha=0)));\ 
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h=[0,hr,(classno(d)/2^t-hr)/2]);\ 

if(mod(d,16)!=mod(4,16)„dd=-d/4;\ 

fdd=factor(dd);\ 

fordiv(dd,ddl,if(ddl>dd/ddl„\ 

beta=l;alpha=l;k=l;\ 

while ( alpha, \ 

if(type(ddl/fdd[k,l])==l,\ 

if(kro(ddl/fdd[k,l],fdd[k,l])==(-l)/^bittest(muu,k-l),\ 

if(k>=matsize(fdd)[l],alpha=0,k=k+l),beta=0;alpha=0),\ 

if(kro((dd/ddl)/fdd[k,l],fdd[k,l])==(-l)^bittest(muu,k-l), \ 

if(k>=matsize(fdd)[l],alpha=0,k=k+l),beta=0;alpha=0)));\ 

hr=hr+beta));t=matsize(fdd)[l]-l;\ 

h=[hr,0,(classno(d)/2^t-hr)/2]);\ 

if(mod(d,16)!=mod(8,16)„dd=-d/4;\ 

if(dd/2==l,hr=l;h=[l,0,0],fdd=factor(dd/2);\ 

fordiv(dd/2,ddl,\ 

beta=l;alpha=l;k=l;\ 

while ( alpha, \ 

if(type(ddl/fdd[k,l])==l,\ 

if(kro(ddl/fdd[k,l],fdd[k,l])==(-l)^bittest(muu,k-l),\ 

if(k>=matsize(fdd)[l],alpha=0,k=k+l),beta=0;alpha=0),\ 

if(kro((dd/ddl)/fdd[k,l],fdd[k,l])==(-l)^bittest(muu,k-l),\ 

if(k>=matsize(fdd)[l],alpha=0,k=k+l),beta=0;alpha=0)));\ 

hr=hr+beta) ;t=matsize(fdd) [1] ;\ 

h=[hr,0,(classno(d)/2^t-hr)/2]));\ 

if(mod(d,16)!=mod(-4,16)„dd=-d/4;\ 

if(dd==l,h=[l,l,0],fdd=factor(dd);\ 

fordiv(dd,ddl,if(ddl>dd/ddl„\ 

beta=l;alpha=l;k=l;\ 

while ( alpha, \ 

if(type(ddl/fdd[k,l])==l,\ 

if(kro(ddl/fdd[k,l],fdd[k,l])==(-l)^bittest(muu,k-l),\ 

if(k>=matsize(fdd)[l],alpha=0,k=k+l),beta=0;alpha=0),\ 

if(kro((dd/ddl)/fdd[k,l],fdd[k,l])==(-l)^bittest(muu,k-l),\ 

if(k>=matsize(fdd)[l],alpha=0,k=k+l),beta=0;alpha=0)));\ 

hrl=hrl+beta) ) ; \ 

fordiv(dd,ddl,if(ddl>dd/ddl„\ 

beta=l;alpha=l;k=l;\ 

while ( alpha, \ 

if(type(ddl/fdd[k,l])==l,\ 

if(kro(2*ddl/fdd[k,l],fdd[k,l])==(-l)^bittest(muu,k-l),\ 

if(k>=matsize(fdd)[l],alpha=0,k=k+l),beta=0;alpha=0),\ 

if(kro(2*(dd/ddl)/fdd[k,l], fdd[k,l])==(-l)^bittest(muu,k-l),\ 

if(k>=matsize(fdd)[l],alpha=0,k=k+l),beta=0;alpha=0)));\ 

hrll=hrll+beta));\ 

hr=hrl+hrll;t=niatsize(fdd) [1] ;\ 

h=[hrI,hrII,(classno(d)/2^t-hr)/2]));h; 
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\\the program h3 

\\hnr(d,et) calculates the number of classes of 
\\non-reflective central symmetries of a 3-dimensional main 
\\hyperbolic lattice with the square-free determinant 
\\d and the invariant et (a non-negative integer) 
\\whose binary decomposition et_{p_k}....et_{p_l} 
Wgives the map of all odd prime divisors p_l,...,p_k 
\\of the d in increasing order to {0,1} 

\r h2 

\\checking the condition (5) =(3.2.7) 

beta5(d,et,n,fd,sfd,alpha,k,b,etap)=\ 

fd=factor(d);sfd=matsize(fd)[l];alpha=l;k=l;\ 

while ( alpha, \ 

if(k>sfd,b=l;alpha=0,\ 

if(type(n/fd[k,l])!=l,k=k+l,\ 

if(fd[k,l]==2,k=k+l,\ 

if(fd[l,l]==2,etap=bittest(et,k-2),etap=bittest(et,k-l));\ 

if(kro(n/fd[k,l],fd[k,l])==(-l)^etap,k=k+l,b=0;alpha=0)))));b; 

\\checking the condition (6) =(3.2.8) 

beta6(d,et,n,fd,sfd,u,k,b)=\ 

fd=factor(d);sfd=matsize(fd)[l];\ 

u=mod(0,8);\ 

for(k=l,sfd,\ 

if(type((d/n)/fd[k,l])!=l„u=u+mod(l-fd[k,l]+4*bittest(et,k-l),8)));\ 

if(u!=mod(-2,8),b=0,b=l);b; 

\\checking the condition (8)=(3.2.10) 

beta8(d,et,n,fd,sfd,u,k,b)=\ 

fd=factor(d);sfd=matsize(fd)[l];\ 

u=mod(0,8);\ 

for(k=l,sfd,\ 

if(type((d/n)/fd[k,l])!=l„u=u+mod(l-fd[k,l]+4*bittest(et,k-l),8)));\ 

u=u+mod(((d/n)^2-l)/2,8);\ 

if(u!=mod(2,8),b=l,b=0);b; 

\\checking the condition (11)=(3.2.13) 

betall(d,et,n,fd,sfd,u,k,b)=\ 

fd=factor(d);sfd=matsize(fd)[l];\ 

u=mod(0,4);\ 

for(k=2,sfd,u=u+mod(l-fd[k,l]+4*bittest(et,k-2),4));\ 

u=-u-mod(l,4);\ 

if(u!=mod(n/2,4),b=0,b=l);b; 

\\calculation of the numbers et_p-|-epsilon(p) if odd p|d 

eps(d,et,e,fd,sfd,sfdl,el,etl,eetl,k)=\ 

if(d==l,e=0,\ 

fd=factor(d);sfd=matsize(fd)[l];\ 

if(fd[l,l]>2,sfdl=sfd,sfdl=sfd-l);\ 

el=vector(sfdl,k,mod((fd[sfd-k+l,l]-l)/2,2));\ 
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eetl=el+etl;eetl=lift(eetl);\ 
e=0;for(k=0,sfdl-l,e=e+2^k*eetl[sfdl-k]));e; 

\\calculation of the numbers et_p+epsilon(p)+omega(p) if odd p|d 

epsomeg(d,et,e,fd,sfd,sfdl,el,etl,eetl,k)=\ 

if(d==l,e=0,\ 

fd=factor(d);sfd=matsize(fd)[l];\ 

if(fd[l,l]>2,sfdl=sfd,sfdl=sfd-l);\ 

el=vector(sfdl,k,mod((fd[sfd-k+l,l]-l)/2+(fd[sfd-k+l,l]^2-l)/8,2));\ 

etl=vector(sfdl,k,mod(bittest(et,sfdl-k),2));\ 

eetl=el+etl;eetl=lift(eetl);\ 

e=0;for(k=0,sfdl-l,e=e+2^k*eetl[sfdl-k]));e; 

\\calculation of the numbers mu_p=eta_p if odd p|t|2d 

muuu(d,et,t,m,fd,sfd,sfdl,kl,k)=\ 

if(di=2,m=0,m=0;fd=factor(d);sfd=matsize(fd)[l];\ 

if(fd[l,l]>2,sfdl=l,sfdl=2);kl=0;\ 

for(k=0,sfd-sfdl,\ 

if(type(t/fd[k+sfdl,l])==l,m=m+2^kl*bittest(et,k);kl=kl+l,)));m; 

\\number of classes of non- reflective central symmetries 

hnr(d,et,h,n,nl)=\ 

h=0;\ 

if(mod(d,2)==mod(l,2),\ 

fordiv(d,n,\ 

if(beta5(d,et,n)==l&&beta6(d,et,n)==l,\ 

h=h+hclass(-d/n,muuu(d,eps(d,et) ,d/n) ) [3] ,) ) ;\ 

fordiv(d,n,\ 

if(beta5(d,et,n)==l&&beta8(d,et,n)==l,\ 

h=h+hclass(-4*d/n,muuu(d,eps(d,et) ,d/n) ) [3] ,) ) ;\ 

fordiv(d,nl,\ 

n=2*nl;\ 

if(beta5(d,et,n)==l,\ 

h=h+hclass(-16*d/n,muuu(d,eps(d,et),d/nl))[3],)),\ 

fordiv(d/2,nl,\ 

n=2*nl;\ 

if(beta5(d,et,n)==l&&betall(d,et,n)==l,\ 

h=h+hclass(-d/n,muuu(d,eps(d,et) ,d/n) ) [3] ,) ) ;\ 

fordiv(d/2,nl,\ 

n=2*nl;\ 

if(beta5(d,et,n)==l&&mod(nl,4)==mod(-d/2,4),\ 

h=h+hclass(-4*d/n,muuu(d,epsomeg(d,et),d/n))[3],));\ 

fordiv(d/2,nl,\ 

n=2*nl;\ 

if(beta5(d,et,n)==l&&mod(nl,4)==mod(d/2,4)&&nid,\ 

h=h+2*hclass(-4*d/n,muuu(d,epsomeg(d,et),d/n))[3]+\ 

hclass(-4*d/n,muuu(d,epsomeg(d,et),d/n))[2],)));h; 

Program 3: refh3 

\\refh3(N) gives the list of invariants (d,et) of 
\\main hyperbolic lattices with the square-free 
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Wnumber h=hnr(d,et)\le 1 of classes of non-reflective 
\\central symmetries. Here et is a non-negative integer 
\\having the binary decomposition et=p_k,...,p_l where 
\\p_l,...,p_k are all odd prime divisors of d in icreasing order 

\r h2 
\r h3 

refh3(m,n,d,fd,sfd,sfdl,sig,n,et,h)=\ 
n=0;\ 

for(d=l,m,\ 
if(issqfree(d)!=l,,\ 

if(d<=2,n=n+l;h=0;et=0;pprint("n=",n," d=",d," et=",et," h=",h),\ 
fd=factor(d);sfd=matsize(fd)[l];\ 
if(fd[l,l]==2,sfdl=sfd-l,sfdl=sfd);\ 
for(et=0,2^sfdl-l,\ 
if(fd[l,l]!=2„\ 

sig=mod(0,8);for(k=2,sfd,sig=sig+mod(l-fd[k,l]+4*bittest(et,k-2),8)));\ 
if(fd[l,l]==2&&(sig==mod(0,8)||sig==mod(-2,8)),\ 
h=hnr(d,et);if(h<=l,n=n-|-l;\ 
pprint("n=",n," d=",d," et=",et," h=",h),)A 
if(fd[l,l]!=2,h=hnr(d,et);\ 

if(h<=l,n=n+l;pprint("n=",n," d=",d," et=",et," h=",h),),)))))) 

Program 4: fundi 1. gen 

\i;\ 

epsilon=l;epsilonl=l;epsilon2=l;n=0;\ 

for(alphal2=l,4,for(alpha23=alphal2,4,\ 

u=((sqrt(2+sqrt(alphal2))+sqrt(2+sqrt(alpha23)))^2-2)^2+0.000001;\ 

for(alphal3=alpha23,u,\ 

if(alpha23==0||issquare(alphal2*alpha23*alphal3)!=l||\ 

-8-|-2*isqrt(alphal2*alpha23*alphal3)4-2*alphal2-|-2*alphal3+2*alpha23<=0,,\ 

alpha=4*idmat(3);alpha[l,2]=alphal2;alpha[2,l]=alphal2;\ 

alpha[2,3]=alpha23;alpha[3,2]=alpha23;alpha[l,3]=alphal3;\ 

alpha[3,l]=alphal3;dalpha=\ 

-8-|-2*isqrt(alphal2*alpha23*alphal3)+2*alphal2+2*alphal3+2>Kalpha23;\ 

a=-2*idmat(3);\ 

fordiv(alpha[l,2],al2,fordiv(alpha[2,3],a23,fordiv(alpha[l,3],al3,\ 

a21=alpha[l,2]/al2;a32=alpha[2,3]/a23;a31=alpha[l,3]/al3;\ 

if(al2*a23*a3l!=a21*al3*a32„\ 

a[l,2]=al2;a[2,l]=a21;a[2,3]=a23;a[3,2]=a32;a[l,3]=al3;a[3,l]=a31;\ 

dd=idmat(3);dd[l,l]=al3*a32;dd[2,2]=a23>Ka31;dd[3,3]=a31*a32;\ 

b=a*dd;b=b/content(b);n=n-|-l;\ 

db=smith(b);r=db[l];\ 

if(r>epsilon,epsilon=r,) ;\ 

fr=factor(r);tfr=matsize(fr)[l];\ 

rl=l;for(j=l,tfr,rl=rl*fr[j,l]);\ 

if(type(rl/2)==l,rl=rl/2,);\ 
if(rl>epsilonl,epsilonl=rl,);\ 

if(tfr<=0„if(fr[tfr,l]>epsilon2,epsilon2=fr[tfr,l],));\ 
^^^^^^^^ ;„j-/'"„ti_" „^ ;„j-/'"„ti_" „;i ^.\ 
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pprint("aIl_l=",epsilonl);pprint("aIl_2=",epsilon2); 

Program 5: fundlO.gen 

\1;\ epsilon=l;epsilonl=l;epsilon2=l;\ 

n=0;\ 

alphal2=0;for(alpha23=l,4,\ 

w=((sqrt(2)+sqrt(2+sqrt(alpha23)))^2-2)^2+0.000001;\ 

for(alphal3=alpha23,w,\ 

if(alpha23==0||issquare(alphal2*alpha23*alphal3)!=l||\ 

-8+2*isqrt(alphal2*alpha23*alphal3)+2*alphal2+2*alphal3+2>Kalpha23<=0,,\ 

alpha=4*idniat(3);alpha[l,2]=alphal2;alpha[2,l]=alphal2;\ 

alpha[2,3]=alpha23;alpha[3,2]=alpha23;alpha[l,3]=alphal3;\ 

alpha[3,l]=alphal3;dalpha=\ 

-8+2*isqrt(alphal2*alpha23*alphal3)+2*alphal2+2*alphal3+2*alpha23;\ 

a=-2*idmat(3);al2=0;a21=0;\ 

fordiv(alpha[2,3],a23,fordiv(alpha[l,3],al3,\ 

a32=alpha[2,3]/a23;a31=alpha[l,3]/al3;\ 

if(al2*a23*a3l!=a21*al3*a32,,\ 

a[l,2]=al2;a[2,l]=a21;a[2,3]=a23;a[3,2]=a32;a[l,3]=al3;a[3,l]=a31;\ 

dd=idmat(3);dd[l,l]=al3*a32;dd[2,2]-a23*a31;dd[3,3]=a31*a32;\ 

b=a*dd;b=b/content(b);n=n+l;\ 

db=smith(b);r=db[l];\ 

if(r>epsilon,epsilon=r,) ;\ 

fr— factor (r);tfr=niatsize(fr)[l];\ 

rl=l;for(j=l,tfr,rl=rl*fr[j,l]);\ 

if(type(rl/2)==l,rl=rl/2,);\ 

if(rl>epsilonl,epsilonl=rl,);\ 

if(tfr<=0„if(fr[tfr,l]>epsilon2,epsilon2=fr[tfr,l],));\ 

))))));pprint("nIO=",n);pprint("aIO=",epsilon);\ 

pprint (" aIO_l =" ,epsilonl ) ;pprint (" aI0_2=" ,epsilon2) ; 

Program 6: fund21.gen 

\i;\ 

epsilon=l;epsilonl=l;epsilon2=l;n=0;\ 

alphal2=alpha23=0;\ 

for(alpha34=l,4,\ 

w=(4*max((sqrt(2)+sqrt(sqrt(alpha34)/4+l/2))^2,\ 

((2+sqrt(sqrt(alpha34)/2+5/4))^2-l/4)/2)-2)^2+0.000001;\ 

for(alphal4=0,w,\ 

for(alphal3=5,36,\ 

if(issquare(u=alphal3*alpha34*alphal4)!=l,,\ 

if(type(alpha24=4+4*(alphal4+alpha34+isqrt(u))/(alphal3-4))!=l||\ 

alpha24<=((sqrt(2)+sqrt(2+sqrt(alpha34)))^2-2)^2-0.0000001„\ 

alpha=4*idmat(4);\ 

alpha[l ,3] =alpha[3, 1] =alphal3;alpha[3,4] =alpha[4,3] =alpha34;\ 

alpha[2,4] =alpha[4,2] =alpha24;alpha[l ,4] =alpha[4, 1] =alphal4;\ 

fordiv(alpha[3,4] ,a34,\ 

fordiv(alpha[l,3],al3,\ 

fordiv(alpha[2,4] ,a24,\ 
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a=-2*idmat(4);\ 

a[3,4] =a34;a[4,3] =alpha[3,4] /a34;\ 

a[2,4]=a24;a[4,2]=alpha[2,4]/a24;\ 

a[l,3]=al3;a[3,l]=alpha[l,3]/al3;\ 

a[4,l]=a41;if(a41==0,a[l,4]=0,a[l,4]=alpha[l,4]/a41);\ 

if(type(a[l,4])!=l„\ 

diag=idmat(4);diag[l,l]=a[l,3]*a[3,4]*a[4,2];\ 

diag[2,2]=a[3,l]*a[4,3]*a[2,4];diag[3,3]=a[3,l]*a[3,4]*a[4,2];\ 

diag[4,4]=a[3,l]*a[4,3]>Ka[4,2];b=a*diag;b=b/content(b);n=n+l;\ 

db=smith(b);r=db[2];\ 

if(r>epsilon,epsilon=r,) ;\ 

fr=factor(r);tfr=niatsize(fr)[l];\ 

rl=l;for(j=l,tfr,rl=rl>Kfr[j,l]);\ 

if(type(rl/2)==l,rl=rl/2,);\ 

if(rl>epsilonl,epsilonl=rl,);\ 

if(tfr<=0„if(fr[tfr,l]>epsilon2,epsilon2=fr[tfr,l],));\ 

))))))))));pprint("nIIl=",n);pprint("aIIl=",epsilon);\ 

pprint (" alll_l=" ,epsilonl) ;pprint (" alll_2=" ,epsilon2) ; 

Program 7: fund20.gen 

\i;\ 

epsilon=l;epsilonl=l;epsilon2=l;n=0;\ 

alphal2=0;alpha23=0;alpha34=0;\ 

for(alphal4=l,287.10,\ 

fordiv(4*alphal4,aa,\ 

if(aa^2>4*alphal4,,alphal3=4+aa;alpha24=4*alphal4/aa+4;\ 

if(alphal3>36,,\ 

alpha=4*idmat(4);\ 

alpha[l,3]=alphal3;alpha[3,l]=alphal3;alpha[3,4]=alpha34;\ 

alpha[4,3] =alpha34;alpha[2,4] =alpha24;alpha[4,2] =alpha24;\ 

alpha[l,4]=alphal4;alpha[4,l]=alphal4;\ 

fordiv(alpha[l,3],al3,fordiv(alpha[2,4],a24,fordiv(alpha[l,4],al4,\ 

a=-2*idmat(4);a[l,3]=al3;a[3,l]=alpha[l,3]/al3;a[l,4]=al4;\ 

a[4,l]=alpha[l,4]/al4;a[l,3]=al3;a[3,l]=alpha[l,3]/al3;\ 

a[2,4] =a24;a[4,2] =alpha[2,4] /a24;\ 

diag=idmat(4);diag[l,l]=a[l,3]*a[l,4]*a[4,2];\ 

diag[2,2]=a[l,3]*a[4,l]*a[2,4];diag[3,3]=a[3,l]*a[l,4]*a[4,2];\ 

diag[4,4]=a[l,3]*a[4,2]*a[4,l];b=a*diag;b=b/content(b);n=n+l;\ 

db=smith(b);r=db[2];\ 

if(r>epsilon,epsilon=r,) ;\ 

fr=factor(r);tfr=niatsize(fr)[l];\ 

rl=l;for(j=l,tfr,rl=rl*fr[j,l]);\ 

if(type(rl/2)==l,rl=rl/2,);\ 

if(rl>epsilonl,epsilonl=rl,);\ 

if(tfr<=0„if(fr[tfr,l]>epsilon2,epsilon2=fr[tfr,l],));\ 

)))))));pprint("nIIO=",n);pprint("aIIO=",epsilon);\ 

pprint (" aIIO_l=" ,epsilonl) ;pprint (" aII0_2=" ,epsilon2) ; 

Program 8: fund30.gen 
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epsilon=l;epsilonl=l;epsilon2=l;n=0;\ 

all2=al23=al34=al45=0;\ 

for(all5=0,900,for(all3=5,36,for(al35=all3,36,\ 

if(issquare(q=all3*al35*all5)==0,,d=(all3+al35+all5-4+isqrt(q))*4;\ 

if(type(all4=d/(al35-4))!=l||all4<=287.108350||type(al25=d/(all3-4))!=l||\ 

al25<=287.108350||\ 

type(al24=(all3*al35+4*all5+4*isqrt(q))>K4/((al35-4)*(all3-4)))!=l||\ 

issquare(ql=all3*al35*al25*al24*all4)==0,,\ 

al=idmat(5)*4;al[l,5]=al[5,l]=all5;al[l,3]=al[3,l]=all3;\ 

al[l,4]=al[4,l]=all4;al[2,4]=al[4,2]=al24;al[2,5]=al[5,2]=al25;\ 

al[3,5]=al[5,3]=al35;\ 

fordiv(all3,al3,\ 

fordiv(al35,a35,a51=isqrt(q)/al3/a35;\ 

if(a51==0,al5==0,if(type(al5=all5/a51)!=l„\ 

fordiv(all4,al4,\ 

fordiv(al24,a24,a52=isqrt(ql)/al3/a35/a24*al4/all4;\ 

if(type(a25=al25/a52)!=l„\ 

a=idmat(5)*-2;\ 

a[l,3]=al3;a[3,l]=all3/al3;\ 

a[l,4]=al4;a[4,l]=all4/al4;\ 

a[l,5]=al5;a[5,l]=a51;\ 

a[2,4]=a24;a[4,2]=al24/a24;\ 

a[2,5]=a25;a[5,2]=a52;\ 

a[3,5]=a35;a[5,3]=al35/a35;\ 

diag=idmat(5);diag[l,l]=a[l,4]*a[l,3]*a[4,2]*a[2,5];\ 

diag[2,2]=a[4,l]*a[l,3]*a[2,4]*a[2,5];\ 

diag[3,3]=a[l,4]*a[3,l]*a[4,2]*a[2,5];\ 

diag[4,4]=a[4,l]*a[l,3]*a[4,2]*a[2,5];\ 

diag[5,5]=a[4,l]*a[l,3]*a[2,4]*a[5,2];\ 

b=a*diag;b=b/content (b) ;n=n+l ; \ 

db=smith(b) ;r=db [3] ;\ 

if(r>epsilon,epsilon=r,) ;\ 

fr=factor(r);tfr=niatsize(fr)[l];\ 

rl=l;for(j=l,tfr,rl=rl*fr[j,l]);\ 

if(type(rl/2)==l,rl=rl/2,);\ 

if(rl>epsilonl,epsilonl=rl,);\ 

if(fr[tfr,l]>epsilon2,epsilon2=fr[tfr,l],);\ 

))))))))))));pprint("nIIIO=",n);\ 

pprint (" aIIIO=" ,epsilon) ;pprint (" aIIIO_l=" ,epsilonl) ;\ 

pprint("aIII0_2=" ,epsilon2); 

Program 9: fund20.niain 

\r h2 
\r h3 

\i;\ 

n=0;\ 

alphal2=0;alpha23=0;alpha34=0;\ 

for(alphal4=l,287.10,\ 
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if(aa^2>4*alphal4,,alphal3=4+aa;alpha24=4*alphal4/aa+4;\ 

alpha=4*idmat(4);\ 

alpha[l,3]=alphal3;alpha[3,l]=alphal3;alpha[3,4]=alpha34;\ 

alpha [4 , 3] =alpha34 ; alpha [2,4] =alpha24 ; alpha [4,2] =alpha24 ; \ 

alpha[l,4]=alphal4;alpha[4,l]=alphal4;\ 

fordiv(alpha[l,3],al3,fordiv(alpha[2,4],a24,fordiv(alpha[l,4],al4,\ 

a=-2*idmat(4);a[l,3]=al3;a[3,l]=alpha[l,3]/al3;a[l,4]=al4;\ 

a[4,l]=alpha[l,4]/al4;a[l,3]=al3;a[3,l]=alpha[l,3]/al3;\ 

a[2,4]=a24;a[4,2]=alpha[2,4]/a24;\ 

diag=idmat(4);diag[l,l]=a[l,3]>Ka[l,4]>Ka[4,2];\ 

diag[2,2]=a[l,3]*a[4,l]*a[2,4];diag[3,3]=a[3,l]*a[l,4]*a[4,2];\ 

diag[4,4]=a[l,3]*a[4,2]*a[4,l];b=a*diag;b=b/content(b);\ 

m=m+l;db=smith(b);dbb=db[2];fdbb=factor(dbb);\ 

if(issqfree(b[l,l])==0||issqfree(b[2,2])==0||issqfree(b[3,3])==0||\ 

issqfree(b[4,4])==0||content([-b[l,l],-b[2,2]])>2||\ 

content([-b[2,2],-b[3,3]])>2||content([-b[3,3],-b[4,4]])>2,,\ 

detb=db[2]*db[3]*db[4];\ 

if(detb==l„\ 

fdetb=factor(detb);\ 

if(content([-b[l,l]*-b[2,2]*-b[3,3]*-b[4,4],16])>l&&\ 

content([-b[l,l]*-b[2,2]*-b[3,3]*-b[4,4],16])<16&&\ 

fdetb[l,l]==2&&mod(fdetb[l,2],2)==mod(l,2)„\ 

gam=0;\ 

for(j=l,matsize(fdetb)[l],\ 

if(fdetb[j,l]!=2&&\ 

type(-b[l,l]*-b[2,2]*-b[3,3]*-b[4,4]/fdetb[j,l])==l&&\ 

mod(fdetb[j,2],2)==mod(0,2),gam=l,));\ 

if(gam==l,,\ 

for(k=0,l,\ 

if(k==0&&content(-[b[l,l]*-b[2,2]*-b[3,3]*-b[4,4],2])==l&&\ 

fdetb[l,l]==2&&mod(fdetb[l,2],2)==mod(l,2)„\ 

if(k==l&&content(-[b[l,l]*-b[2,2]*-b[3,3]*-b[4,4],2])==l,\ 

b=2*b;db=smith(b);dbb=db[2];fdbb=factor(dbb);gaml=0,gaml=l);\ 

if(k==l&&gaml=l„\ 

detb=db [2] *db [3] *db [4] ;fdetb=factor (detb) ; \ 

d=l;for(k=l,matsize(fdetb)[l],\ 

if(mod(fdetb[k,2],2)==mod(l,2),d=d*fdetb[k,l],));\ 

if(d<=2,et=0,\ 

fd=factor(d);if(fd[l,l]==2,dl=d/2,dl=d);fdl=factor(dl);\ 

et=0;for(k=l,matsize(fdl)[l],\ 

if(type(b[l,l]/fdl[k,l])==l,\ 

if(kro(b[l,l]/fdl[k,l],fdl[k,l])==l„et=et+2^(k-l)),\ 

if(type(b[2,2]/fdl[k,l])==l,\ 

if(kro(b[2,2]/fdl[k,l],fdl[k,l])==l„et=et+2^(k-l)),\ 

if(kro(d*b[l,l]*b[2,2]/fdl[k,l],fdl[k,l])==l„et=et+2^(k-l))))));\ 

hhh=hnr(d,et);if(hhh>l,,\ 

n=n+l;pprint("n=",n);pprint(a);pprint(b);pprint(db);\ 

pprint(fdbb);pprint("d=",d," et=",et," h=",hhh);\ 
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aaa=0;nnn=niatsize(u) [1] ;\ 
for(j=l,nnn,if(u[j,]==[d,et,hhh],aaa=l,));\ 
if(aaa==l,,ul=matrix(nnn+l,3,j,k,0);\ 
for(t=l,nnn,\ 

if(u[t,l]<d||(u[t,l]==d&&u[t,2]<et),ul[t,]=u[t,],\ 

ul[t+l,]=u[t,]));\ 

for(t=l,nnn+l,if(ul[t,]==[0,0,0],ul[t,] = [d,et,hhh],));u=ul));\ 

))))))))))))));PP^i'^t("i^="7i^);PP^i'^t("'^^tsize u=",matsize(u)); 

Program 10: reflO.l 

\\niain(n,h) calculates for U+<-2n> (it is given by the matrix g) 

\\the vectors v of the height \le h from P(\M)_{pr} 

\\and calculates chains e and f and their Gram matrices ge, gf 

refl(n,h,m,ml,j,k,vl,yl,y2,y3,z)=\ 

g=[0,l,0;l,0,0;0,0,-2*n];\ 

m=3;v=matrix(m,3,j,k,0);v[l,] = [0,0,l];v[2,] = [n,0,-l];v[3,] = [-l,l,0];\ 

for(hl=2,h,\ 

fordiv(hl,y2,if(type(y2^2/hl)!=l||type(n*hl/y2^2)!=l„\ 

d=-2>Ky2^2/hl;\ 

for(z=floor(sqrt((2*y2^2-d)/(2*n))),y2,\ 

y3=-z;yl=(2>Kn>Ky3^2+d)/(2>Ky2);\ 

if(type(2*yl/d)!=l||content([yl,y2,y3])!=l„\ 

u=[yl,y2,y3];\ 

alpha=l;ml=l;\ 

while(alpha,if(nil>m,alpha=0,if(v[ml,]*g*u~>=0,ml=mH-l,alpha=0));\ 

if(ml<=m,,m=ml;vl=matrix(m,3,j,k,0);\ 

for(j=l,m-l,vl[j,]=v[j,]);vl[m,] = [yl,y2,y3];v=vl;kill(vl)\ 

)))))));v; 

Wgetting el-matrix from v 
elfromv(v,s,sl,alpha,s2,ex)=\ 

s=matsize(v)[l];\ 

el=matrix(s,3,j,k,0);el[l,]=v[2,];el[2,]=v[l,];\ 

alpha=l;sl=2;\ 

while ( alpha, \ 

if(sl==s,alpha=0,\ 

for(j=l,s,\ 

if(v[j,]==el[sl-l,]||v[j,]==el[sl,]||\ 

(v[j,]*g*el[sl,n^2/((v[j,]*g*v[j,n*(el[sl,]*g*el[sl,n)>l,A 

s2=sl+l;el[s2,]=v[j,]));if(s2>sl,sl=s2,alpha=0)));\ 

ex=matrix(sl,3,j,k,el[j,k]);el=ex; 

Wgetting e2-matrix from v 

e2fromv(v,s,sl,alpha,s2,ex)=\ 

s=matsize(v)[l];\ 

e2=matrix(s,3,j,k,0);e2[l,]=v[l,];e2[2,]=v[2,];\ 

alpha=l;sl=2;\ 

while ( alpha, \ 

if(sl==s,alpha=0,\ 
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if(v[j,]==e2[sl-l,]||v[j,]==e2[sl, 

(v[j,]*g*e2[sl,n^2/((v[j,]*g*v[j,n*(e2[sl,]*g*e2[sl,n)>l,A 

s2=sl+l;e2[s2,]=v[j,]));if(s2>sl,sl=s2,alpha=0)));\ 

ex=matrix(sl ,3,j ,k,e2 [j ,k] ) ;e2=ex; 

Wgetting e-matrix from v 

efromv(v,el,e2,s,sl,s2)=\ 

s=matsize(v) [1] ;sl=matsize(el) [1] ;s2=matsize(e2) [1] ;\ 

if(sl==s&&s2==s,e=el,\ 

e=matrix(sl+s2-2,3,j,k,0);\ 

for(j=l,s2,e[j,]=e2[s2+l-j,]);\ 

for(j=l,sl-2,e[s2+j,]=el[2+j,]));\ 

e;ge=e*g*e ; 

Wgetting fl -matrix from v 

flfromv(v,e,s,sl,s2,q,alpha,flx)=\ 

s=matsize(v) [1] ;sl=matsize(e) [1] ;\ 

if(s==sl,fl=0;vl=0,\ 

fl=matrix(s-sl,3,j,k,0);vl=v;\ 

for(j=l,s,\ 

for(k=l,sl,\ 

if(vl[j,]!=e[k,]„vl[j,] = [0,0,0])));\ 

q=l;s2=0;alpha=l;\ 

while ( alpha, \ 

if(q>s,alpha=0,\ 

if(vl[q,]==[0,0,0],q=q+l,s2=s2+l;fl[s2,]=vl[q,];\ 

vl[q,] = [0,0,0];alpha=0)));\ 

if(s2==s-sl,,\ 

for(t=l,s,\ 

for(q=l,s,\ 

if(vl[q,]==[0,0,0]„\ 

if((vl[q,]*g*fl[s2,n^2/((vl[q,]*g*vl[q,n*(fl[s2,]*g*fl[s2,n)>l,A 

s2=s2+l;fl[s2,]-vl[q,];vl[q,]-[0,0,0])))));\ 

if(s2==0„flx=matrix(s2,3,j,k,fl[j,k]);fl=flx);fl); 

Wgetting f2-matrix from v 

f2fromv(fl,s,sl,s2,s3,q,f2x)=\ 

if(fl==0,f2=0;s3=0,\ 

s=matsize(vl) [1] ;sl=matsize(e) [1] ;s2=matsize(fl) [1] ;\ 

f2=matrix(s-sl-s2+l,3,j,k,0);\ 

s3=l;f2[l,]=fl[l,];\ 

for(t=l,s,\ 

for(q=l,s,\ 

if(vl[q,]==[0,0,0]W 

if((vl[q,]*g*f2[s3,n^2/((vl[q,]*g*vl[q,n*(f2[s3,]*g*f2[s3,n)>l,A 

s3=s3+l;f2[s3,]=vl[q,];vl[q,] = [0,0,0]))));\ 

if (s3==0„f2x=matrix(s3,3,j ,k,f2 [j ,k] ) ;f2=f2x) ) ;f2; 

Wgetting f-matrix from v 

ffromv(fl,f2,sl,s2)=\ 

if(f2==0,f=flA 

sl=matsize(fl) [1] ;s2=matsize(f2) [1] A 
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f=matrix(sl+s2-l,3,j,k,0);\ 

for(j=l,s2,f[j,]=f2[s2+l-j,]);\ 
for(j=l,sl-l,f[s2+j,]=fl[l+j,])));\ 

if(f-=o,gf=o,gf=f*g*r); 

\i;\ 

niain(n,h)=refl(n,h);elfroniv(v);e2froniv(v,el);efroniv(v,el,e2);\ 
flfroniv(v,e);f2froniv(fl);ffroniv(fl,f2); 

Program 11: refl0.12 

\\main(nl,n2,n3,h) calculates for <nl>\oplus <-n2>\oplus <-n3> 

\\(it is given by the matrix g) 

\\the vectors v of the height \le h from P(\M)_{pr} 

\\and calculates chains e and f and their Gram matrices ge, gf 

refl(nl,n2,n3,h,m,ml,j,k,vl,yl,y2,y3,z,d,dd,hl,u,ul,w,wl)=\ 

g=[nl,0,0;0,-n2,0;0,0,-n3];\ 

m=2;v=matrix(m,3,j,k,0);v[l,] = [0,l,0];\ 

if(n2==n3,v[2,] = [0,-l,l],v[2,] = [0,0,l]);\ 

for(hl=l,h,\ 

fordiv(hl,yl,dd=gcd(2*lcm(lcm(nl,n2),n3),2*nl*yl);\ 

fordiv(dd,d,if(hl!=2*nl*yl^2/d„\ 

for(z=0,floor(sqrt((w=nl*yl^2+d)/n2)+0.000001),\ 

if(type(2>Kn2>Kz/d)!=l||type(wl=(w-n2*z^2)/n3)!=l„\ 

if(issquare(wl)!=l,,\ 

y2=-z;y3=-isqrt(wl);if(type(2*n3>Ky3/d)!=l||content([yl,y2,y3])!=l,,\ 

u=[yl,y2,y3];\ 

alpha=l;ml=l;\ 

while(alpha,if(nil>m,alpha=0,if(v[ml,]>Kg>Ku~>=0,ml=mH-l,alpha=0));\ 

if(ml<=m,,m=ml;vl=matrix(m,3,j,k,0);\ 

for(j=l,m-l,vl[j,]=v[j,]);vl[m,] = [yl,y2,y3];v=vl;kill(vl)\ 

))))))))));v; 

This part is the same as in Program 10: reflO.l between \\** and \\*** 

\i;\ 

main(nl,n2,n3,h)=\ 

refl(nl,n2,n3,h);elfromv(v);e2fromv(v,el);efromv(v,el,e2);\ 

flfromv(v,e);f2fromv(fl);ffromv(fl,f2); 

Program 12: refl0.13 

\\main(nl,n2,n3,epsl,eps2,eps3,h) calculates for 

\\<nl>\oplus <-n2>\oplus <-n3>(epsl/2,eps2/2,eps3/2) 

\\(it is given by the matrix g) where ni\ge and epsi=0 or 1 and 

\\either all of them are or at least two of them are not zero, 

\\the vectors v of the height \le h from P(\M)_{pr} 

\\and calculates chains e and f and their Gram matrices ge, gf 

refl(nl,n2,n3,epsl,eps2,eps3,h,m,ml,j,k,vl,yl,y2,y3,ylt,y2t,y3t,\ 

z,d,dt,dd,hl,u,ul,w,wl)=\ 

g=[nl,0,0;0,-n2,0;0,0,-n3];eps=[epsl,eps2,eps3]/2;\ 
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if(n2/n3!=l&&n2/n3!=3&&n2/n3!=l/3,v[l,] = [0,l,0];v[2,] = [0,0,l],\ 

if(n2==n3,if(eps==[0,0,0]||eps==[l/2,l/2,l/2],v[l,] = [0,l,0];v[2,] = [0,-l,l],\ 
if(eps==[0,l/2,l/2],v[l,] = [0,l,0];v[2,] = [0,-l/2,l/2],\ 

v[l,] = [0,l,0];v[2,] = [0,0,l]))A 
if(n2/n3==3,if(eps==[0,l/2,l/2],v[l,] = [0,l,0];v[2,] = [0,-l/2,l/2],\ 

v[l,]-[0,l,0];v[2,] = [0,0,l])A 
if(n2/n3==l/3,if(eps==[0,l/2,l/2],v[l,] = [0,0,l];v[2,] = [0,l/2,-l/2],\ 

v[l,] = [0,l,0];v[2,] = [0,0,l]),))));\ 
for(hl=l,h,\ 

fordiv(hl,ylt,dd=gcd(2>Klcm(lcm(nl,n2),n3),nl>Kylt);\ 

fordiv(dd,d,if(d*hl!=nl*ylt^2,,dt=4*d;\ 

for(z=0,floor(sqrt((w=nl*ylt^2+dt)/n2)+0.000001),\ 

if(type(n2*z/d)!=l||type(wl=(w-n2*z^2)/n3)!=l„\ 

if(issquare(wl)!=l,,\ 

y2t=-z;y3t=-isqrt(wl);if(type(n3*y3t/d)!=l,,\ 

yl=ylt/2;y2=y2V2;y3=y3t/2;u=[yl,y2,y3];\ 

if(type(2*u*g*eps /d)!=l,,\ 

if(mod(2*u,2)!=mod([0,0,0],2)&&mod(2*u,2)!=mod(2*eps,2)„\ 

if((mod(2*u,2)==mod([0,0,0],2)&&(content(u)>l||\ 

mod(2*u,4)==mod(4*eps,4))) 1 1\ 

(eps!=[0,0,0]&&mod(2*u,2)==mod(2*eps,2)&&content(2*u)>l)„\ 

alpha=l;nil=l;\ 

while(alpha,if(nil>ni,alpha=0,if(v[nil,]*g*u~>=0,nil=niH-l,alpha=0));\ 

if(nil<=ni,,ni=nil;vl=niatrix(ni,3,j,k,0);\ 

for(j=l,m-l,vl[j,]=v[j,]);vl[m,] = [yl,y2,y3];v=vl;kill(vl)\ 

)))))))))))));v; 

This part is the same as in Program 10: reflO.l between \\** and \\*** 

\i;\ 

main(nl,n2,n3,epsl,eps2,eps3,h)=\ 
refl(nl,n2,n3,epsl,eps2,eps3,h);\ 
elfromv(v);e2fromv(v,el);efromv(v,el,e2);\ 
flfromv(v,e);f2fromv(fl);ffromv(fl,f2); 
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